This is maybe the most contentious point in my argument, and I agree this is not at all guaranteed to be true, but I have not seen MIRI arguing that it’s overwhelmingly likely to be false.
Hm? I feel this is basically the single argument they makes in the whole first third of the book. “You don’t get what you train for” et cetera. I think they’d disagree current LLMs are aligned, like at all, and getting ASIs “about as aligned as current LLMs” would get us all killed instantly.
I think this is what you should argue against in a post like this. The brain-emulations and collective intelligence do no heavy lifting. Ironically I’ve head Eliezer on several occasions literally propose “getting the smartest humans, uploading them, running them for a thousand subjective years” as a good idea.
For the record: I think their argument is coherent, but doesn’t provide the level of confidence they display. I’d put like ~50% on “If anyone builds it, with anything remotely like current techniques, everyone dies”. Maybe 75% if a random lab does it under intense time pressure, and like 25% if a safety conscious international project did it, with enough time to properly/thoroughly/carefully implement all the best prosaic safety techniques, but without enough time to make any new really fundamental approaches or changes to how the AIs are created.
Hm? I feel this is basically the single argument they makes in the whole first third of the book. “You don’t get what you train for” et cetera. I think they’d disagree current LLMs are aligned, like at all, and getting ASIs “about as aligned as current LLMs” would get us all killed instantly.
I think this is what you should argue against in a post like this. The brain-emulations and collective intelligence do no heavy lifting. Ironically I’ve head Eliezer on several occasions literally propose “getting the smartest humans, uploading them, running them for a thousand subjective years” as a good idea.
For the record: I think their argument is coherent, but doesn’t provide the level of confidence they display. I’d put like ~50% on “If anyone builds it, with anything remotely like current techniques, everyone dies”. Maybe 75% if a random lab does it under intense time pressure, and like 25% if a safety conscious international project did it, with enough time to properly/thoroughly/carefully implement all the best prosaic safety techniques, but without enough time to make any new really fundamental approaches or changes to how the AIs are created.