I mostly expect by the time we know how to make a seed superintelligence and give it a particular utility function… well, first of all the world has probably already ended, but second of all I would expect progress on corrigibility and such to have been made and probably to present better avenues.
If Omega handed me aligned-AI-part-2.exe, I’m not quite sure how I would use it to save the world? I think probably trying to just work on the utility function outside of a simulation is better, but if you are really running out of time then sure, I guess you could try to get it to simulate humans until they figure it out. I’m not very convinced that referring to a thing a person would have done in a hypothetical scenario is a robust method of getting that to happen, though?
to me it kind of is; i mean, if you have that, what do you do then? how do you use such a system to save the world?
I mostly expect by the time we know how to make a seed superintelligence and give it a particular utility function… well, first of all the world has probably already ended, but second of all I would expect progress on corrigibility and such to have been made and probably to present better avenues.
If Omega handed me
aligned-AI-part-2.exe
, I’m not quite sure how I would use it to save the world? I think probably trying to just work on the utility function outside of a simulation is better, but if you are really running out of time then sure, I guess you could try to get it to simulate humans until they figure it out. I’m not very convinced that referring to a thing a person would have done in a hypothetical scenario is a robust method of getting that to happen, though?