A magical genie shows up at your bedroom window. Nobody has bothered to align this genie.
Would you try asking this genie for the solution to AI Alignment?
Your answer to this question should mirror your response to many proposals for Godzilla Strategies.
I don’t think it would make any meaningful difference whether I asked or not. In this hypothetical, there’s an unaligned magical genie on the loose! I probably won’t ask it anything, and hope that it goes away instead of responding to whatever sleep-addled nonsense I happen to think up on the spot.
If I asked it to solve anything, it would probably be the genie alignment problem. AI alignment is a potential future existential risk, while an unaligned being of immense magical power is a definite right now existential risk.
It’s just going to do whatever it’s inclined to do anyway, but maybe there’s a microscopic chance that “whatever it’s inclined to do” happens to be “what I’d like it to do when I ask”. It almost certainly won’t be, but I figure that maybe there’s a better chance than many of the other things I might do in response to a genie showing up at my bedroom window.