AI users never find out they haven’t “got it”.
There’s a certain genre of “educational” material that leads to similar outcomes as described in this article. Sometimes I enjoy outsourcing my thinking to YouTube channels like PBS SpaceTime and Veritasium and, if I’m not careful, I can fool myself into thinking I know more about quantum mechanics or gravity waves than I actually do.
It turns out that learning things for real is hard, and things that feel “comfortable” or “passive” should be met with skepticism for their actual educational value. The tricky thing is that we like being comfortable, and so we often inflate the educational value of such experiences.
Steps an Aligned AI Should Take:
Keep all current humans alive, in a safe manner, in such a way that causes minimal value drift. This step may or may not require comas (must be safe).
Create authentic simulations of the human brain on a computer.
Run simulated societies with the digital brains on the computer. Test various utopias on the digital brains, collect data from the tests and obtain feedback from the digital brains themselves. Certain ethical codes should govern the simulated environments to avoid excessive suffering and allow digital brains continued existence (e.g., allowing versions of step 4 and 5).
Present the data to the alive humans in the AI’s base reality after a reasonable number of trials from step 3. If comas were used in step 1, we would awaken the humans in this step.
Permit humans the chance to experience different utopias—via simulation or otherwise—but require such humans to leave their inhabited utopias at set intervals (e.g. every 5 to 100 years) such that they have the chance to re-evaluate the utopia experience from prior held value-frames in a neutral space. Of course, the humans should be allowed to freely exit one utopia for a different utopia, unless such humans (after careful review) committed to living in a particular utopia for the full duration prior to the required interval.
There’s lots of clean up that would need to be done (e.g., what is a neutral space?, what is reasonable?, how do we make sure digital brains are treated ethically). But I think this is where I land in terms of what I imagine to lead to very good outcomes.