Cultural divides like the one described here have created major slowdowns in other movements and sciences. They have caused sciences to remain wrong for decades. This must not happen in alignment. This post does vital work in communicating across that divide. It treats both perspectives with sympathy, a critical piece missing from much scientific debate.
I think this is one of the most important posts yet written. How could that be, you ask? It doesn’t even try to solve the technical alignment problem, and surely that’s the most important problem!
The solution that matters isn’t the best one: it’s the one that’s actually tried. And there’s a cultural divide on what types of solutions might work. This post sums up the debate that will determine how we roll the dice on the future. It does a remarkably good job of summing up the perspectives and intuitions that are most common, even though it’s couched as satire.
Pessimists hold that current alignment techniques have little chance of working. Optimists think that the apparent relatively good alignment of current LLMs means we’re on track to succeed at alignment. The arguments are complex and seem to depend on intuitions and worldviews (see my short list of major Cruxes of disagreement on alignment difficulty if you like, although there are a few more I’d like to update it with at this point.
This post is the one I reference most often to describe the critical cultural divide between alignment optimists and pessimists (and probably most overall).
Cultural divides like the one described here have created major slowdowns in other movements and sciences. They have caused sciences to remain wrong for decades. This must not happen in alignment. This post does vital work in communicating across that divide. It treats both perspectives with sympathy, a critical piece missing from much scientific debate.
I think this is one of the most important posts yet written. How could that be, you ask? It doesn’t even try to solve the technical alignment problem, and surely that’s the most important problem!
The solution that matters isn’t the best one: it’s the one that’s actually tried. And there’s a cultural divide on what types of solutions might work. This post sums up the debate that will determine how we roll the dice on the future. It does a remarkably good job of summing up the perspectives and intuitions that are most common, even though it’s couched as satire.
Pessimists hold that current alignment techniques have little chance of working. Optimists think that the apparent relatively good alignment of current LLMs means we’re on track to succeed at alignment. The arguments are complex and seem to depend on intuitions and worldviews (see my short list of major Cruxes of disagreement on alignment difficulty if you like, although there are a few more I’d like to update it with at this point.
This post is the one I reference most often to describe the critical cultural divide between alignment optimists and pessimists (and probably most overall).