I’m leaving this review primarily because this post somehow doesn’t have one yet, and it’s way too important to get dropped out of the Review!
ELK had some of the most alignment community engagement of any technical content that I’ve seen. It is extremely thorough, well-crafted, and aims at a core problem in alignment. It serves as an examplar of how to present concrete problems to induce more people to work on AI alignment.
That said, I personally bounced after reading the first few pages of the document. It was good as far as I got, but it was pretty effortful to get through, and (as mentioned above) already had tons of attention on it.
FWIW I think the Eliciting Latent Knowledge problem doesn’t stand well on its own as an introduction, and thinking about this problem is way better when you see the bigger picture that Paul is working through and used to generate it, in his post My Research Methodology (my review here). That walks through multiple of the major steps in Paul’s reasoning that led to this problem being brought up, rather than just dumping you in it, and is written in Paul’s native voice. I’d rank that post as substantially more useful than this one.
I’m leaving this review primarily because this post somehow doesn’t have one yet, and it’s way too important to get dropped out of the Review!
ELK had some of the most alignment community engagement of any technical content that I’ve seen. It is extremely thorough, well-crafted, and aims at a core problem in alignment. It serves as an examplar of how to present concrete problems to induce more people to work on AI alignment.
That said, I personally bounced after reading the first few pages of the document. It was good as far as I got, but it was pretty effortful to get through, and (as mentioned above) already had tons of attention on it.
FWIW I think the Eliciting Latent Knowledge problem doesn’t stand well on its own as an introduction, and thinking about this problem is way better when you see the bigger picture that Paul is working through and used to generate it, in his post My Research Methodology (my review here). That walks through multiple of the major steps in Paul’s reasoning that led to this problem being brought up, rather than just dumping you in it, and is written in Paul’s native voice. I’d rank that post as substantially more useful than this one.