Someone may think “Anomalous worlds imply the simulation-runners will save us from failing at alignment!”
My reply is: Why are they running a simulation where we have to solve alignment?
At a first pass, if we’re in a simulation, it’s probably for research, rather than e.g. a video game or utopia. (H/t an IRL friend for pointing this out).
Therefore, if we observe ourselves needing to solve AI alignment (and not having solved it yet), the simulation-runners potentially also need AI alignment to get solved. And if history is any guide, we should not rely on any such beings “saving us” before things cross a given threshold of badness.
(There are other caveats I can respond to about this, but please DM me about them if you think of them, since they may be infohazard-leaning and (thus) should not be commented publicly, pending more understanding.)
Someone may think “Anomalous worlds imply the simulation-runners will save us from failing at alignment!”
My reply is: Why are they running a simulation where we have to solve alignment?
At a first pass, if we’re in a simulation, it’s probably for research, rather than e.g. a video game or utopia. (H/t an IRL friend for pointing this out).
Therefore, if we observe ourselves needing to solve AI alignment (and not having solved it yet), the simulation-runners potentially also need AI alignment to get solved. And if history is any guide, we should not rely on any such beings “saving us” before things cross a given threshold of badness.
(There are other caveats I can respond to about this, but please DM me about them if you think of them, since they may be infohazard-leaning and (thus) should not be commented publicly, pending more understanding.)