It’s not obvious to me that personal physical beauty (as opposed to say, beauty in music or mathematics or whatever) isn’t negative sum. Obviously beauty in any form can be enjoyable, but we describe people as “enchantingly beautiful” when a desire to please or impress them distorts our thinking, and if this effect isn’t purely positional it could be bad. Conventionally beautiful people are also more difficult to distinguish from one another.
There’s also the meta-aesthetic consideration that I consider it ugly to pour concern into personal physical beauty, either as a producer or consumer, but it’s unclear how widespread such a preference/taste is. (I would consider a world where everyone was uglier because they spent less time on it to be a much more beautiful world; but clearly many people disagree, for instance George Orwell in 1984 seems to find it distasteful and degrading that the Party encourages its members to have a functional, less dolled-up personal appearance.)
The main utility of suppressing ideas is suppressing the ability to coordinate around them. If a lot of people hold some latent antisemitic ideas, but anybody expressing explicit antisemitism is regarded as a sort of loathesome toad, that prevents the emergence of active antisemitic politics, even if it’s a wash in terms of changing any minds (suppose, plausibly, that conservation of expected evidence means that “why can’t you say this” more or less balances out people being exposed to fewer arguments).
Obviously there are plenty of costs as well—enforcement mechanisms can be weaponized for other purposes, preference falsification also makes it more difficult to identify the good guys, etc. Your original contrarian take is still a largely defensible one, though really I think the nature of the internet is such that it’s kind of a fait accompli under current conditions that it’s harder to make things taboo and prevent the coordination of your opponents.