Not sure what makes you think ‘strawmen’ at 2, but I can try to unpack this more for you.
Many warnings about unaligned AI start with the observation that it is a very bad idea to put some naively constructed reward function, like ‘maximize paper clip production’, into a sufficiently powerful AI. Nowadays on this forum, this is often called the ‘outer alignment’ problem. If you are truly worried about this problem and its impact on human survival, then it follows that you should be interested in doing the Hard Thing of helping people all over the world write less naively constructed reward functions to put into their future AIs.
John writes:
Far and away the most common failure mode among self-identifying alignment researchers is to look for Clever Ways To Avoid Doing Hard Things. [...] The most common pattern along these lines is to propose outsourcing the Hard Parts to some future AI [...]
This pattern of outsourcing the Hard Part to the AI is definitely on display when it comes to 2 above. Academic AI/ML research also tends to ignore this Hard Part entirely, and implicitely outsources it to applied AI researchers, or even to the end users.
Not sure what makes you think ‘strawmen’ at 2, but I can try to unpack this more for you.
Many warnings about unaligned AI start with the observation that it is a very bad idea to put some naively constructed reward function, like ‘maximize paper clip production’, into a sufficiently powerful AI. Nowadays on this forum, this is often called the ‘outer alignment’ problem. If you are truly worried about this problem and its impact on human survival, then it follows that you should be interested in doing the Hard Thing of helping people all over the world write less naively constructed reward functions to put into their future AIs.
John writes:
This pattern of outsourcing the Hard Part to the AI is definitely on display when it comes to 2 above. Academic AI/ML research also tends to ignore this Hard Part entirely, and implicitely outsources it to applied AI researchers, or even to the end users.