The book takes a risk by—and I assume it is intentional—ignoring some of the more nuanced arguments (esp. your Tricky hypothesis 2). I think they are trying to shock the Overton Window to the very real risk of death by alignment failure if society continues with business as usual. The risk management seems to be:
A) Yet another carefully hedged warning call (like this one). Result:
95% few people update, but the majority continues business as usual.
5% brings the topic over the tipping point.
B) If Anyone Builds It, Everyone Dies. Result:
50% the topic becomes a large discussion point, the Overton Window includes the risk.
50% critical voices point out technical weaknesses of part 3 and the effort fizzles out.
If these numbers are halfway right, B seems advisable? And you can still do A if it fails!
I think Buck and Eliezer both agree you should only say shocking things if they were true. I think if Eliezer believed what Buck believes, he would have found a title that was still aimed at the overton-smashing strategy but honest.
I don’t think you are arguing only about the title. Titles naturally have to simplify, but the book content has to support it. The “with techniques like those available today” in “If anyone builds it (with techniques like those available today), everyone dies” sure is an important caveat, but arguably it is the default. And, as Buck agrees, the authors do qualify it that way in the book. You don’t have to repeat the qualification each time you mention it.
The core disagreement doesn’t seem to be about that but about leaving out Tricky hypothesis 2. I’m less sure that is an intentional omission by the authors. Yudkowsky sure has argued many times that alignment is tricky and hard and may feel that the burden of proof is on the other side now.
(caveat: I’m still reading the book)
The book takes a risk by—and I assume it is intentional—ignoring some of the more nuanced arguments (esp. your Tricky hypothesis 2). I think they are trying to shock the Overton Window to the very real risk of death by alignment failure if society continues with business as usual. The risk management seems to be:
A) Yet another carefully hedged warning call (like this one). Result:
95% few people update, but the majority continues business as usual.
5% brings the topic over the tipping point.
B) If Anyone Builds It, Everyone Dies. Result:
50% the topic becomes a large discussion point, the Overton Window includes the risk.
50% critical voices point out technical weaknesses of part 3 and the effort fizzles out.
If these numbers are halfway right, B seems advisable? And you can still do A if it fails!
I think Buck and Eliezer both agree you should only say shocking things if they were true. I think if Eliezer believed what Buck believes, he would have found a title that was still aimed at the overton-smashing strategy but honest.
I don’t think you are arguing only about the title. Titles naturally have to simplify, but the book content has to support it. The “with techniques like those available today” in “If anyone builds it (with techniques like those available today), everyone dies” sure is an important caveat, but arguably it is the default. And, as Buck agrees, the authors do qualify it that way in the book. You don’t have to repeat the qualification each time you mention it.
The core disagreement doesn’t seem to be about that but about leaving out Tricky hypothesis 2. I’m less sure that is an intentional omission by the authors. Yudkowsky sure has argued many times that alignment is tricky and hard and may feel that the burden of proof is on the other side now.