I was definitely struck by the resemblance between her notion of “normative dependence” and the ideas behind the CIRL framework. And I think that the fix to the AI reasoning about something more intelligent is more or less the same thing humans do, which is we abstract away the planning and replace it with some “power” to do something. Like if I imagine playing Magnus Carlsen in chess, I don’t simulate a chess game at all, I compare an imaginary chess-winning power I attribute to us in my abstracted mental representation.
But as for the philosophical problems she mentions in the interview, I felt like they fell into pretty standard orthodox philosophical failure modes. For the sake of clarity, I guess I should say I mean the obsession with history, and the default assumption that questions have one right answer—things I think are boondoggles have to be addressed just because they’re historical, and there’s too much worry about what humans “really” are like as opposed to consideration of models of humans.
Thanks! This is an interesting recommendation.
I was definitely struck by the resemblance between her notion of “normative dependence” and the ideas behind the CIRL framework. And I think that the fix to the AI reasoning about something more intelligent is more or less the same thing humans do, which is we abstract away the planning and replace it with some “power” to do something. Like if I imagine playing Magnus Carlsen in chess, I don’t simulate a chess game at all, I compare an imaginary chess-winning power I attribute to us in my abstracted mental representation.
But as for the philosophical problems she mentions in the interview, I felt like they fell into pretty standard orthodox philosophical failure modes. For the sake of clarity, I guess I should say I mean the obsession with history, and the default assumption that questions have one right answer—things I think are boondoggles have to be addressed just because they’re historical, and there’s too much worry about what humans “really” are like as opposed to consideration of models of humans.