Why I think it’s relevant: I think it’s pretty clear that there is a lot of good reason to put EY near the provably safe AI camp.
Look, I am really confident that seeing the stuff from the provably safe AI camp fills Eliezer with the same kind of frustration as becomes me when I see it. I don’t get what the provably safe AI camp people are talking about, and I don’t think Eliezer gets it either (or like, maybe he understands the psychology better than I do, but I really doubt he believes it).
other reasons include his critiques of modern ML methods as being particularly bad for alignment and general AI pessimism.
I think modern ML methods are particularly bad for alignment. I do not think this has anything to do with thinking that we should “prove AI safe”. I do think an appropriate approach would include many mathematical proofs because mathematical proofs are one among a large set of tools you bring to bear to solve complicated problems, in the same way that of course many mathematical proofs were involved in landing rockets on the moon. The fact that people don’t seem to understand that you want to use mathematical proofs for anything but “proving that systems are safe end to end” or things like that is what Eliezer’s rant was about.[1]
Like, maybe it’s wrong, but it is a pretty understandable type of wrong.
I personally find the position that if you utilize proofs in your thinking, then you must be attempting to prove end-to-end things about a complicated real-world system, at the level of complexity of “proving that this rocket will land on the moon” or “proving that this self-driving car system will never cause any crashes” is very silly. IDK whether it’s “understandable”, but I think it deserves being solidly rebuked.
Which is not to say that it spells out what those things are, saying explicitly that it doesn’t go into that because it’s not the main topic of the essay, but I assure you, they exist.
I share your confusion at the idea of “mathematically proving AI safe” haha. This convo has made me realize I’ve conflated alignment pessimists in general with the provably safe AI people in particular too much in my mind.
Look, I am really confident that seeing the stuff from the provably safe AI camp fills Eliezer with the same kind of frustration as becomes me when I see it. I don’t get what the provably safe AI camp people are talking about, and I don’t think Eliezer gets it either (or like, maybe he understands the psychology better than I do, but I really doubt he believes it).
I think modern ML methods are particularly bad for alignment. I do not think this has anything to do with thinking that we should “prove AI safe”. I do think an appropriate approach would include many mathematical proofs because mathematical proofs are one among a large set of tools you bring to bear to solve complicated problems, in the same way that of course many mathematical proofs were involved in landing rockets on the moon. The fact that people don’t seem to understand that you want to use mathematical proofs for anything but “proving that systems are safe end to end” or things like that is what Eliezer’s rant was about.[1]
I personally find the position that if you utilize proofs in your thinking, then you must be attempting to prove end-to-end things about a complicated real-world system, at the level of complexity of “proving that this rocket will land on the moon” or “proving that this self-driving car system will never cause any crashes” is very silly. IDK whether it’s “understandable”, but I think it deserves being solidly rebuked.
Which is not to say that it spells out what those things are, saying explicitly that it doesn’t go into that because it’s not the main topic of the essay, but I assure you, they exist.
I share your confusion at the idea of “mathematically proving AI safe” haha. This convo has made me realize I’ve conflated alignment pessimists in general with the provably safe AI people in particular too much in my mind.