Unfortunately, it's just a sad fact of life.
Thankfully, my spouse and I both work at home, so no dealing with backbiting co-workers.
And neither one of us live close to our families, so we don't have to deal with them either. Not that our families are bad or any such, but we both just like staying at home, doing our thing and staying out of everyone's way.
We wouldn't have it any other way.