It is indeed conceivable to construct “safe” oracle AIs that answer mathematical questions. See also writeup by Jim Babcock and my comment. The problem is that the same technology can be relatively easily repurposed into an agent AI. Therefore, anyone building an oracle AI is really bad news unless FAI is created shortly afterwards.
I think that oracle AIs might be useful to control the initial testing process for an (agent) FAI but otherwise are far from solving the problem.
It is indeed conceivable to construct “safe” oracle AIs that answer mathematical questions. See also writeup by Jim Babcock and my comment. The problem is that the same technology can be relatively easily repurposed into an agent AI. Therefore, anyone building an oracle AI is really bad news unless FAI is created shortly afterwards.
I think that oracle AIs might be useful to control the initial testing process for an (agent) FAI but otherwise are far from solving the problem.