Loved this post! I’ve been thinking about some tangential ideas lately but probably won’t end up writing something myself. Here are a few other ways I’ve found that randomness can benefit science:
Randomness may result in faster algorithms for some computations
I don’t (yet!) have the background to understand the mathematics behind this quanta article, but apparently randomness can provide algorithmic speedup for computations as orderly/un-random seeming as linear systems. I don’t think it has been proven that the current fastest algorithm for this computation (which uses randomness in its solution) is the best possible, but this solution seems to point in that direction.
Also consider Shor’s algorithm’s speedup on finding prime numbers, which is inherently random.
Acting out of sync with respect to outside patterns is usually best (especially when you want accurate measurements of averages).
A medieval lord wants to know about how much food each of his serfs have throughout the year to ensure they are not overstuffed nor starving. However, he has so many serfs he can only visit each one once a year. If he checked each serf the same day each year, the springtime-checked serf would always have a lot less food than the fall-checked serf due to the time of harvest, even if there was no real difference between them. A much better approach would be to randomize which day you see each serf every year.
Averages of large group’s guesses are much better than most people’s individual guesses
See this article about how accurate group average guesses are at guessing the number of Jellybeans in a jar / more applicably how accurate market predictions guess the real value of a company. Importantly, when the independence of individual’s guess was broken (i.e. the students had time to talk to each other about the number of Jellybeans in the jar), the average guess became much worse.
Thanks for catching the grammar mistake—fixed! These are interesting extensions of the basic idea of using more randomness in science, thanks for sharing. Your last point makes me think about the use of prediction markets to guess which studies will replicate, something that people have successfully done.
Loved this post! I’ve been thinking about some tangential ideas lately but probably won’t end up writing something myself. Here are a few other ways I’ve found that randomness can benefit science:
Randomness may result in faster algorithms for some computations
I don’t (yet!) have the background to understand the mathematics behind this quanta article, but apparently randomness can provide algorithmic speedup for computations as orderly/un-random seeming as linear systems. I don’t think it has been proven that the current fastest algorithm for this computation (which uses randomness in its solution) is the best possible, but this solution seems to point in that direction.
Also consider Shor’s algorithm’s speedup on finding prime numbers, which is inherently random.
Acting out of sync with respect to outside patterns is usually best (especially when you want accurate measurements of averages).
A medieval lord wants to know about how much food each of his serfs have throughout the year to ensure they are not overstuffed nor starving. However, he has so many serfs he can only visit each one once a year. If he checked each serf the same day each year, the springtime-checked serf would always have a lot less food than the fall-checked serf due to the time of harvest, even if there was no real difference between them. A much better approach would be to randomize which day you see each serf every year.
Also see Cicada’s, which brood for 17 years before hatching to get out of sync with the life cycle of predators/competitors.
Averages of large group’s guesses are much better than most people’s individual guesses
See this article about how accurate group average guesses are at guessing the number of Jellybeans in a jar / more applicably how accurate market predictions guess the real value of a company. Importantly, when the independence of individual’s guess was broken (i.e. the students had time to talk to each other about the number of Jellybeans in the jar), the average guess became much worse.
See also the Group Rationality and Efficient Market Hypothesis tags on Less Wrong.
On a grammar note, this sentence isn’t finished in the article: “(e.g. the gods speak to us through the , yet...”
Thanks for catching the grammar mistake—fixed! These are interesting extensions of the basic idea of using more randomness in science, thanks for sharing. Your last point makes me think about the use of prediction markets to guess which studies will replicate, something that people have successfully done.
https://www.pnas.org/content/112/50/15343