I fully admit that I do not have strong outside-view evidence that this method will objectively improve your rationality—if I did, I would post it. But many (most?) rationality techniques discussed here lack such evidence as well.
This seems like something that should be fixed. A few ideas:
Scientific studies—seems to slow, requires dealing with academia… I think we can do better.
Well designed self-experimentation—Gwern’s studies on nootropics are the best examples I know of, but there are others like [Seth Robert’s] self experiments.
Studies done by a more formal organization—For example, I think CFAR might be doing studies like this.
Regarding my second point, it seems that this sort of thing could benefit a lot from a division of labor—where a small group of people design the experiments, and many more people just follow the instructions. It might be worth trying to organize a group of people willing to participate in these sorts of experiments, so that it is easier to test rationality techniques.
This seems like something that should be fixed. A few ideas:
Scientific studies—seems to slow, requires dealing with academia… I think we can do better.
Well designed self-experimentation—Gwern’s studies on nootropics are the best examples I know of, but there are others like [Seth Robert’s] self experiments.
Studies done by a more formal organization—For example, I think CFAR might be doing studies like this.
Regarding my second point, it seems that this sort of thing could benefit a lot from a division of labor—where a small group of people design the experiments, and many more people just follow the instructions. It might be worth trying to organize a group of people willing to participate in these sorts of experiments, so that it is easier to test rationality techniques.