Even for an ideal reasoner, successful retrospective predictions clearly do not play the same role as prospective predictions. The former must inevitably be part of locating the hypothesis; they thus play a weaker role in confirming it. Eliezer’s story you link to is about how the “traditional science” dictum about not using retrospective predictions can be just reversed stupidity; but just reversing young Eliezer’s stupidity in the story one more time doesn’t yield intelligence.
Edit: this comment has been downvoted, and in considering why that may be, I think there’s ambiguities in both “ideal reasoner” and “play the same role”. Yes, the value of evidence does not change depending on when a hypothesis was first articulated, so some limitless entity that was capable of simultaneously evaluating all possible hypotheses would not care. However, a perfectly rational but finite reasoner could reasonably consider some amount old evidence to have been “used up” in selecting the hypothesis from an implicit background of alternative hypotheses, without having to enumerate all of those alternatives; and thus habitually avoid recounting a certain amount of retrospective evidence. Any “successful prediction” would presumably be by a hypothesis that had already passed this threshold (otherwise it’s just called a “lucky wild-ass guess”). I’m speaking in simple heuristic terms here, but this could be made more rigorous and numeric, up to and including a superhuman level I’d consider “ideal”.
Even for an ideal reasoner, successful retrospective predictions clearly do not play the same role as prospective predictions. The former must inevitably be part of locating the hypothesis; they thus play a weaker role in confirming it. Eliezer’s story you link to is about how the “traditional science” dictum about not using retrospective predictions can be just reversed stupidity; but just reversing young Eliezer’s stupidity in the story one more time doesn’t yield intelligence.
Edit: this comment has been downvoted, and in considering why that may be, I think there’s ambiguities in both “ideal reasoner” and “play the same role”. Yes, the value of evidence does not change depending on when a hypothesis was first articulated, so some limitless entity that was capable of simultaneously evaluating all possible hypotheses would not care. However, a perfectly rational but finite reasoner could reasonably consider some amount old evidence to have been “used up” in selecting the hypothesis from an implicit background of alternative hypotheses, without having to enumerate all of those alternatives; and thus habitually avoid recounting a certain amount of retrospective evidence. Any “successful prediction” would presumably be by a hypothesis that had already passed this threshold (otherwise it’s just called a “lucky wild-ass guess”). I’m speaking in simple heuristic terms here, but this could be made more rigorous and numeric, up to and including a superhuman level I’d consider “ideal”.