Absence of a sufficiently concrete description of how to accomplish a task is Bayesian update towards the task is impossible: absence of evidence IS evidence of absence. I never said I know for certain that any plan CAN’T work, what I am saying is that those plans people are coming up with are not even close to work. They think they are having ideas on how to finish the world, they are not, they are just imperfect plans that can go wrong for many reasons no matter how clever you are, don’t guarantee human extinction and most importantly, give us a considerable time window in which we could use an AGI to solve the alignment problem for future AGIs. EY et al. do not even consider this a possibility not because an AGI won’t be able to solve the alignment problem, but because the AGI would kill us all first. If you realiz that this far from proven, that path to AGI safety becomes way more believable
Absence of a sufficiently concrete description of how to accomplish a task is Bayesian update towards the task is impossible: absence of evidence IS evidence of absence. I never said I know for certain that any plan CAN’T work, what I am saying is that those plans people are coming up with are not even close to work. They think they are having ideas on how to finish the world, they are not, they are just imperfect plans that can go wrong for many reasons no matter how clever you are, don’t guarantee human extinction and most importantly, give us a considerable time window in which we could use an AGI to solve the alignment problem for future AGIs. EY et al. do not even consider this a possibility not because an AGI won’t be able to solve the alignment problem, but because the AGI would kill us all first. If you realiz that this far from proven, that path to AGI safety becomes way more believable