A physics research team has members who can (and occasionally do) in secret insert false signals into the experiment the team is running. The goal is practice resistance to false positives. A very interesting approach, first time I’ve heard about physicists using it.
Bias combat in action :-)
The LIGO is almost unique among physics experiments in practising ‘blind injection’. A team of three collaboration members has the ability to simulate a detection by using actuators to move the mirrors. “Only they know if, and when, a certain type of signal has been injected,”...
Two such exercises took place during earlier science runs of LIGO, one in 2007 and one in 2010. … The original blind-injection exercises took 18 months and 6 months respectively. The first one was discarded, but in the second case, the collaboration wrote a paper and held a vote to decide whether they would make an announcement. Only then did the blind-injection team ‘open the envelope’ and reveal that the events had been staged.
Wait, I’m confused. How does this practice resistance to false positives? If the false signal is designed to mimic what a true detection would look like, then it seems like the team would be correct to identify it as a true detection. I feel like I’m missing something here.
I don’t know the details, but the detection process is essentially statistical and very very noisy. It’s not a “we’ll know it when we see it” case, it’s more like “out of the huge number of wiggles and wobbles that we have recorded, what can’t we explain and therefore might be a grav wave”.
I would guess one of the points is that a single observation is unreliable in a high-noise environment.
A physics research team has members who can (and occasionally do) in secret insert false signals into the experiment the team is running. The goal is practice resistance to false positives. A very interesting approach, first time I’ve heard about physicists using it.
Bias combat in action :-)
Source
Wait, I’m confused. How does this practice resistance to false positives? If the false signal is designed to mimic what a true detection would look like, then it seems like the team would be correct to identify it as a true detection. I feel like I’m missing something here.
I don’t know the details, but the detection process is essentially statistical and very very noisy. It’s not a “we’ll know it when we see it” case, it’s more like “out of the huge number of wiggles and wobbles that we have recorded, what can’t we explain and therefore might be a grav wave”.
I would guess one of the points is that a single observation is unreliable in a high-noise environment.
This is really fascinating, I wonder what other existing big science efforts ‘blind injection’ would benefit.