I find that in reading this I end up with a better understanding of Paul and John (to the extent John summarized Paul well, but it feels right). Double-crux feels like a large ask given the hunt for a mutually shared counterfactual change that just seems like a lot to identify; ideological turing test means someone putting aside who they are and what they think too much; but “deltas” feel like a nice alternative that’s not as complicated to compute and doesn’t lose ones reference to what they think.
I’d be pretty into everyone for six months being gung-ho on deltas and trying to identify and flesh them out. It’s a cool approach. For that matter, I’d be excited for someone to try and flesh out a methodology for eliciting deltas akin to the effort Eli Tyre once put into double-crux, as an adjacent approach.
What really stuck for me from this one is the point Eliezer makes in his top-level comment:
I think that the AI’s internal ontology is liable to have some noticeable alignments to human ontology w/r/t the purely predictive aspects of the natural world; it wouldn’t surprise me to find distinct thoughts in there about electrons. As the internal ontology goes to be more about affordances and actions, I expect to find increasing disalignment.
The Natural Abstraction Hypothesis really does feel so plausible when thinking about predicting the external world, as the external world does appear to have all this structure you’d expect any mind to parse. Because of Eliezer’s response comment (and I give a lot of credit to posts that elicit valuable comments), I see we have a problem anyway.
I’d have liked if @johnswentworth has responded to Eliezer’s comment at more length, though maybe he did and I missed? Still, giving this post a 4 in the review, same as the other.
I’d have liked if @johnswentworth has responded to Eliezer’s comment at more length, though maybe he did and I missed? Still, giving this post a 4 in the review, same as the other.
The review I have of this post is much the same as I left oh John’s other delta post, My AI Model Delta Compared To Christiano:
What really stuck for me from this one is the point Eliezer makes in his top-level comment:
The Natural Abstraction Hypothesis really does feel so plausible when thinking about predicting the external world, as the external world does appear to have all this structure you’d expect any mind to parse. Because of Eliezer’s response comment (and I give a lot of credit to posts that elicit valuable comments), I see we have a problem anyway.
I’d have liked if @johnswentworth has responded to Eliezer’s comment at more length, though maybe he did and I missed? Still, giving this post a 4 in the review, same as the other.
Done.