A few thoughts on this:
The person advocating such experiments is usually an advocate of the change rather than a curious seeker of truth. Consequently, the lack of evaluation is a feature not a bug.
It is very hard to get good data out of any experiment that has a sample size of one.
It is often very hard to measure the actual thing you desire, and so as a consequence, insofar as there is measurement it is measurement of something that can be measured rather than something that is a useful measurement.
Even insofar as it is possible to measure something useful, it is often impossible to get all experimenters to agree on which of the useful metrics to use.
To some degree these things can be dealt with honestly by taking an A/B approach. So, for example
You have an open office plan on the first floor, and keep it as is on the second floor.
You introduce new forum rules for the month of December, and revert on January (and see if the forum members demand the return on the new rules.)
You spin off a new forum: my-fourm-expanded, that allows in the new users and see if the old users naturally migrate to the broader discussion. Failure and the new forum will automatically die.
However, as you say in your comment oftentimes such things are introduced more out of an implementation of a belief system about what is “right”, rather than a curious investigation into which is more effective. It often requires utterly disastrously bad results before reverting the change is likely.
I have read neither the book nor the chapter you refer to, but I wll comment on fake beliefs: they are evidently beneficial, or else they would not exist, and I think that is because the goal of humans is not the achievement of rationality, but other things (Maslow, for example). Rationality may lead to those benefits, but it is only one way.
Let me offer you two specific examples:
The belief that the earth was created 6,000 years ago
Here in the USA a disturbingly large percentage of people hold this belief, something that is plainly not true, since all the evidence in earth and life science point to this being an underestimate of nearly 1 millionth of the correct value.
So why do so many people hold that belief. I think there is good reason to do so from a simple cost benefit analysis. On the one hand what exactly is the cost of this belief? Almost no decision you make in your day to day life depends on the correct value of the age of the earth. I think the primary down side is that you might be mocked, or held in low esteem for such a believe. But let me return to that in a moment.
What is the benefit of such a belief? Well people believe that because the Bible and the teachers of the Bible say so. So by conforming to this believe they have access to religion and the benefits of religion. What are those? A lovely social group to be part of. A caring community to support you. An outlet for your charitable instincts. A pre-created moral code to help you resolve challenging moral issues. A community for your children that you think trustworthy and a good place to learn good things. It gives you a sense that you are not some bit of ephemeral fluff in the universe, but are significant and important. And a eschatological view that both promises long term victory and that gives you comfort in both the death of loved ones and of yourself.
As to the concern about “being held in low esteem”, one of the brilliant features of religion is that this, which seems a negative, can be made into a positive, specifically the sense of “persecution” which drives you more strongly to your group attachment. It further emphasizes your “special-ness”, as one of the persecuted ones.
Nobody who is not religious (Christian mainly, but probably other Abrahamic religions too—I’m not knowledgeable enough to comment) holds this view, but many who are do, because, quite rationally, the benefits outweigh the costs by a quite considerable amount.
The attraction of women to “bad boys”
This is of course a gross generalization, but there is no doubt that some cohort of women tend to be attracted to “bad boys”. Why is that? I think in this case a rather different calculation is going on, this time much lower in the cognitive stack.
The downside of a bad boy is manifest—he is more likely to treat her poorly (that for example, a worshipful “nice guy”, something that is often unattractive to women), he is more likely to be violent against her, he is more likely to abandon her, and he is more likely to cheat on her.
But what is the upside? I think it is a combination of two things. On the one hand there are secondary characteristics to the bad boy, namely confidence, strength, dominance and miscellaneous other “alpha” attributes. These are attractive because the driving force here is sexual attraction, and the innate desire of the woman to pass on to her progeny genes that will maximize their reproductive capabilities (that, after all, at the bottom line in our deep inner brain, is that driving force in sexual congress). However, there is a secondary thing, namely that the non alpha male, who offers the other side to reproductive success, namely the facility and resources to nurture the child, is readily available to her since the majority of non alpha males have less sexual access, and so must be willing to commit (and may prefer to commit) to get that access.
However, this all operates at a far lower level than the conscious mind. No doubt we have all experienced a longing for a partner that we know on a rational level is a very poor choice, but our deep inner longings often drive us off course.
Since this alpha male/beta male/female thing can be controversial, let me suggest that another example of our biological urges overcoming our rational is one we all encounter—namely the urge to eat food that we know is unhealthy for us. It can be argued that food considered unhealthy is only unhealthy in the high availability, long living, famine free world we inhabit. but our bodies are not evolved for such an environment and so crave foods more suited to the ancestral human condition. Here again, our biology drives us away from rational choices to satisfy something much deeper.
In Conclusion
So I have offered here two chains where rationality is not the main goal. On the one hand the secondary benefits of an irrational choice might outweigh the benefits of a rational choice. One the other our assumption that rationality drives our decisions is often at cross purposes with the real deep biological drives that influence a lot more of our choices that we might imagine.