Throw me in the "who gives a shit?" pile. If you really care that much if your group knows you're charming, do what I do in those rare situations: cast lev on your pet.
It's really more about the group bitching than anything personal. NPCs look cool imo. Anyways, higher lvl groups know about it, so i guess it's not much of a big deal anymore. Just takes a little getting used to for a few people.