1. I think the AIs might be philosophically competent enough to solve ~all the problems, and using the AIs to solve them is basically the right move. We try to make this clear at the start of the epilogue. We wanted to be somewhat more concrete in the scenario than meta level solutions like this, and try to sketch out in concrete detail what the solutions might actually look like, which is why we didn’t just stop at “handoff to the AIs”, though I do think in practice we should mostly be handing off to the AIs at this point. 2. I agree that making the AIs this philosophically competent may be very difficult and not happen by default. I think this is indeed a big concern, and I wish we’d written (and thought) about this more carefully. The place we’ve written the most about this is here: https://ai-2040.com/supplements/alignment-roadmap#phase-4-handoff (which TBC is written from a Plan C perspective, assuming much less lead time than Plan A, and is the supplement from which we linked your post). In Plan A, the story is largely that we have a huge amount of time with ~human level AIs, and many people have access for many years, and those people will (hopefully) make progress on this.
Thanks for the comment!
1. I think the AIs might be philosophically competent enough to solve ~all the problems, and using the AIs to solve them is basically the right move. We try to make this clear at the start of the epilogue. We wanted to be somewhat more concrete in the scenario than meta level solutions like this, and try to sketch out in concrete detail what the solutions might actually look like, which is why we didn’t just stop at “handoff to the AIs”, though I do think in practice we should mostly be handing off to the AIs at this point.
2. I agree that making the AIs this philosophically competent may be very difficult and not happen by default. I think this is indeed a big concern, and I wish we’d written (and thought) about this more carefully. The place we’ve written the most about this is here: https://ai-2040.com/supplements/alignment-roadmap#phase-4-handoff (which TBC is written from a Plan C perspective, assuming much less lead time than Plan A, and is the supplement from which we linked your post). In Plan A, the story is largely that we have a huge amount of time with ~human level AIs, and many people have access for many years, and those people will (hopefully) make progress on this.