If you're going to say something uncomplimentary it's probably the safest play to do it.
Had a comical experience one time when I walked up on a group at work to find that the guy with his back turned was talking about me. Everybody else knew I was there and he turned around, saw me, and stopped in mid sentence. So I said, "What did I do then, Arthur?" And I'll give him full marks because he went on and finished what he was saying. We all had a good laugh. But, with hindsight, I wonder if the others laughing at him, or at me?