here is a list of reasons I have previously written for why the Singularity might never happen.
That being said, EY’s primary argument that alignment is impossible seems to be “I tried really hard to solve this problem and haven’t yet.” Which isn’t a very good argument.
I could be wrong, but my impression is that Yudkowski’s main argument isn’t right now about the technical difficulty of a slow program creating something aligned, but mainly about the problem of coordinating so that nobody cuts corners while trying to get there first (I mean of course he has to believe that alignment is really hard, and that it is very likely for things that look aligned to be unaligned for this to be scary).
here is a list of reasons I have previously written for why the Singularity might never happen.
That being said, EY’s primary argument that alignment is impossible seems to be “I tried really hard to solve this problem and haven’t yet.” Which isn’t a very good argument.
I could be wrong, but my impression is that Yudkowski’s main argument isn’t right now about the technical difficulty of a slow program creating something aligned, but mainly about the problem of coordinating so that nobody cuts corners while trying to get there first (I mean of course he has to believe that alignment is really hard, and that it is very likely for things that look aligned to be unaligned for this to be scary).