> perhaps implying an expectation and inner consciousness on the part of the so-called “agent”
I’m curious if you think this passage is also mistaken, or if it is correctly describing a real problem with current trajectories. EY usually doesn’t bring up consciousness because it is not a crux for him, but I wonder if you think he’s wrong in this recent time that he did bring it up.
I didn’t mean to claim that this “consciousness” insinuation has or is messing up this community’s reasoning about AI alignment, just that the insinuation exists—and to train the skill of spotting possible mistakes before (and not after) they occur.
I do think that “‘expectation’ insinuates inner beliefs” matters, as it helps prop up the misconception of “agents maximize expected reward” (by adding another “supporting detail” to that story).
I didn’t mean to claim that this “consciousness” insinuation has or is messing up this community’s reasoning about AI alignment, just that the insinuation exists—and to train the skill of spotting possible mistakes before (and not after) they occur.
I do think that “‘expectation’ insinuates inner beliefs” matters, as it helps prop up the misconception of “agents maximize expected reward” (by adding another “supporting detail” to that story).