Secondly, the possibility that human values may not converge. Yudkowsky considered CEV obsolete almost immediately after its publication in 2004. He states that there’s a “principled distinction between discussing CEV as an initial dynamic of Friendliness, and discussing CEV as a Nice Place to Live” and his essay was essentially conflating the two definitions.
But I totally agree that CEV is a useful concept to have. Also Yudkowsky’s later writing (like the Arbital post presumably around 2016) should trump his earlier take in 2004. Or maybe the meaning of CEV shifted a bit over the years from sth more specific to a very indirect pointer. Idk, I don’t remember the original CEV paper well.
Hrmm, what a weird description. The full quote of what it’s quoting is:
Once we have something that approximates a volition of the human species, that volition then has the chance to write its own superintelligence, optimization process, legislative procedure, god, or constitution. I try not to get caught up on CEV as a model of the actual future, even though it seems like a Nice Place To Live. The purpose of CEV as an initial dynamic is not to be the solution, but to ask what solution we want.
I really don’t know where that “and his essay was essentially conflating the two definitions” part comes from.
There is another LW wikitag here, which includes:
But I totally agree that CEV is a useful concept to have. Also Yudkowsky’s later writing (like the Arbital post presumably around 2016) should trump his earlier take in 2004. Or maybe the meaning of CEV shifted a bit over the years from sth more specific to a very indirect pointer. Idk, I don’t remember the original CEV paper well.
Hrmm, what a weird description. The full quote of what it’s quoting is:
I really don’t know where that “and his essay was essentially conflating the two definitions” part comes from.