This doesn’t really add anything to the discussion, and certainly doesn’t provide any solutions, but I find it satisfying to view the alignment problem as a special case of ethics.
Well, I do think it’s worth noting that (granting the Biblical frame) God failed to align humans, even a chosen subset of humans, despite complete control of our source code, complete control over our initial conditions, and the ability to intervene in any way at any time to provide additional training data. And that both rule and virtue based approaches are insufficient given a mind whose underlying structure is not already inclined towards the ‘right’ shape. And that AFAICT even if we generalize to see all religions as different attempts at alignment of humans, the result does not typically look like a balance of forces that results in overall human flourishing and moral progress, though it sometimes does achieve that. And that at higher levels of capabilities our adherence to the original alignment restrictions changes and typically decreases.
Well, I do think it’s worth noting that (granting the Biblical frame) God failed to align humans, even a chosen subset of humans, despite complete control of our source code, complete control over our initial conditions, and the ability to intervene in any way at any time to provide additional training data. And that both rule and virtue based approaches are insufficient given a mind whose underlying structure is not already inclined towards the ‘right’ shape. And that AFAICT even if we generalize to see all religions as different attempts at alignment of humans, the result does not typically look like a balance of forces that results in overall human flourishing and moral progress, though it sometimes does achieve that. And that at higher levels of capabilities our adherence to the original alignment restrictions changes and typically decreases.
I thought all of these were obvious and well known. But yes, all of these are things I was pointing at.
Probably just me being oblivious. I wrote my comment when I was very tired.