humans talking to each other already has severe misalignment. ownership exploitation is the primary threat folks seem to fear from ASI: “you’re made of atoms the ai can use for something else” ⇒ “you’re made of atoms jeff bezos and other big capital can use for something else”. I don’t think point 1 holds strongly. youtube is already misaligned; it’s not starkly superhuman, but it’s much better at selecting superstimuli than most of its users. hard asi would amplify all of these problems immensely, but because they aren’t new problems, I do think seeking formalizations of inter-agent safety is a fruitful endeavor.
humans talking to each other already has severe misalignment. ownership exploitation is the primary threat folks seem to fear from ASI: “you’re made of atoms the ai can use for something else” ⇒ “you’re made of atoms jeff bezos and other big capital can use for something else”. I don’t think point 1 holds strongly. youtube is already misaligned; it’s not starkly superhuman, but it’s much better at selecting superstimuli than most of its users. hard asi would amplify all of these problems immensely, but because they aren’t new problems, I do think seeking formalizations of inter-agent safety is a fruitful endeavor.
Oh I agree with all that. I said “it helps a lot for their alignability” not “they are all aligned.”
makes sense, glad we had this talk :thumbsup: