I wish that the sequence was there to read. However, even this would benefit from doublechecking for mistakes like mocking people for trying to ban open-source models. The case against open-sourced models is that they lack safeguards unless they are as capable of goal-guarding as Agent-4, thus allowing terrorists to use open-source models to hack into important systems or to create bioweapons.
The framework also has a more severe issue. Your most recent quick take mentioned that “AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA”. Suppose, by an oversimplified analogy, that alignment research required $10B, but would receive the following dependent on which political party AI governance chooses and who wins the elections:
Party
Dems win
Reps win
AI governance allies with Dems
$1B
0
AI governance allies with Reps
0
$100M
In order for AI governance to choose the Reps, the Reps would have to be tenfold more likely to win, and even then the situation would be grim. Of course, the real world has many more strategic options, but I doubt that they could have drastically change the game.
I wish that the sequence was there to read. However, even this would benefit from doublechecking for mistakes like mocking people for trying to ban open-source models. The case against open-sourced models is that they lack safeguards unless they are as capable of goal-guarding as Agent-4, thus allowing terrorists to use open-source models to hack into important systems or to create bioweapons.
The framework also has a more severe issue. Your most recent quick take mentioned that “AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA”. Suppose, by an oversimplified analogy, that alignment research required $10B, but would receive the following dependent on which political party AI governance chooses and who wins the elections:
Party
Dems win
Reps win
AI governance allies with Dems
$1B
0
AI governance allies with Reps
0
$100M
In order for AI governance to choose the Reps, the Reps would have to be tenfold more likely to win, and even then the situation would be grim. Of course, the real world has many more strategic options, but I doubt that they could have drastically change the game.
I’ve added a footnote explaining my position on open-weight models in more detail.