If the AI is deliberately making things happen in the world, then I would say it’s not an Oracle, it’s an Agent whose I/O channel happens to involve answering questions. (Maybe the programmers intended to make an Oracle, but evidently they failed!)
“If you have an aligned Oracle, then you wouldn’t ask it to predict unpredictable things. Instead you would ask it to print out plans to solve problems—and then it would come up with plans that do not rely on predicting unpredictable things.”
If the AI is deliberately making things happen in the world, then I would say it’s not an Oracle, it’s an Agent whose I/O channel happens to involve answering questions. (Maybe the programmers intended to make an Oracle, but evidently they failed!)
My response to @Jeffrey Heninger would have been instead:
“If you have an aligned Oracle, then you wouldn’t ask it to predict unpredictable things. Instead you would ask it to print out plans to solve problems—and then it would come up with plans that do not rely on predicting unpredictable things.”