I overall agree with the “People are doing gymnastics when just the straightforward thing works” and the “people, if given the facts reasonably, don’t need to be persuaded that AI extinction risk is bad, they conclude that easily” frames. I give Bayes points for making the bet that it’ll work and having reasonable results come from the bet—good work!
My disagreement comes with the “thus everything others are doing is cope/avoidance” framing. I think a good physicist entering the field should most likely go into interp work, a good economist is likely best suited to thinking through TAI impacts and figuring out what we should prepare for, and people with good people/oratory skills and situational awareness/understanding of the problem should talk to politicians. I think that if the field has not done what it has done, you would find it a lot harder to talk to politicians! AI frontier labs could say “well, there’s a lot we didn’t try yet, we don’t know if it’s dangerous” and you can instead counteract that with “look at all the stuff we tried, look at all the danger we found, look at how things fail” and feel confident in your claims. You can point to economist work (since recently, and increasingly) and say “these are models of what happens if the companies continue their work” and make a case for a pause/stop much better than without them, when accelerationist demagoguery can claim business as usual. Spectre is one failure mode, but myopia to ones preferred work is another. The problem is wicked, and affects many areas, and experts should do their expertise in different fields if we want to have a good shot to make it out.
I overall agree with the “People are doing gymnastics when just the straightforward thing works” and the “people, if given the facts reasonably, don’t need to be persuaded that AI extinction risk is bad, they conclude that easily” frames. I give Bayes points for making the bet that it’ll work and having reasonable results come from the bet—good work!
My disagreement comes with the “thus everything others are doing is cope/avoidance” framing. I think a good physicist entering the field should most likely go into interp work, a good economist is likely best suited to thinking through TAI impacts and figuring out what we should prepare for, and people with good people/oratory skills and situational awareness/understanding of the problem should talk to politicians. I think that if the field has not done what it has done, you would find it a lot harder to talk to politicians! AI frontier labs could say “well, there’s a lot we didn’t try yet, we don’t know if it’s dangerous” and you can instead counteract that with “look at all the stuff we tried, look at all the danger we found, look at how things fail” and feel confident in your claims. You can point to economist work (since recently, and increasingly) and say “these are models of what happens if the companies continue their work” and make a case for a pause/stop much better than without them, when accelerationist demagoguery can claim business as usual. Spectre is one failure mode, but myopia to ones preferred work is another. The problem is wicked, and affects many areas, and experts should do their expertise in different fields if we want to have a good shot to make it out.