Could we budget our trust in a predictor by keeping track of how hard we’ve tried to maximize predicted approval? Let Arthur expect any action to have a chance to get its approval overestimated, and he will try proposing fewer alternatives. Just like when frequentists decrease p-value thresholds as they ask more questions of the same data. To avert brainwashing the real Hugh, assume that even asking him is just a predictor of the “true” approval function.
Could we budget our trust in a predictor by keeping track of how hard we’ve tried to maximize predicted approval? Let Arthur expect any action to have a chance to get its approval overestimated, and he will try proposing fewer alternatives. Just like when frequentists decrease p-value thresholds as they ask more questions of the same data. To avert brainwashing the real Hugh, assume that even asking him is just a predictor of the “true” approval function.