Thinking about (innate drives → valenced world states → associated states → learned drives → increasingly abstract valenced empowerment) brings up for me this question of seeking a very specific world state with high predicted valence & empowerment. And this I feel like is accurately described, but awkward to think about, from the frame of Jacob’s W/P/U/V/A distinction. Like how it’s accurate but difficult to think about biology from the frame of movements of protons and electrons. I think if we zoom in on the W/P plan making portion and adopt a different frame, we see a consequentialist plan generator that does directed search through projected futures based on W (world model). And this then is rather like Eliezer’s Outcome Pump. If you zoom out, the Outcome Pump is one part of an agent. It’s only in the zoomed in view that you see a non-sentient search process that searches for valenced empowerment over extrapolations made from running simulations of the World Model. I’d argue that something very like this planning process is occuring in AlphaZero and DeepNash (stratego). But those have narrow world models, and search systems designed to work over narrow world models.
Consider again the Tragedy of Group Selectionism: Some early biologists asserted that group selection for low subpopulation sizes would produce individual restraint in breeding; and yet actually enforcing group selection in the laboratory produced cannibalism, especially of immature females. It’s obvious in hindsight that, given strong selection for small subpopulation sizes, cannibals will outreproduce individuals who voluntarily forego reproductive opportunities. But eating little girls is such an un-aesthetic solution that Wynne-Edwards, Allee, Brereton, and the other group-selectionists simply didn’t think of it. They only saw the solutions they would have used themselves.
So, notice this idea that humans are doing an aesthetically guided search. Seems to me this is an accurate description of human thought / planning. I think this has a lot of overlap with the aesthetic imagining of a nice picture being done by Stable Diffusion or other image models. And overlap with using Energy Models to imagine plans out of noise.
Thinking about (innate drives → valenced world states → associated states → learned drives → increasingly abstract valenced empowerment) brings up for me this question of seeking a very specific world state with high predicted valence & empowerment. And this I feel like is accurately described, but awkward to think about, from the frame of Jacob’s W/P/U/V/A distinction. Like how it’s accurate but difficult to think about biology from the frame of movements of protons and electrons. I think if we zoom in on the W/P plan making portion and adopt a different frame, we see a consequentialist plan generator that does directed search through projected futures based on W (world model). And this then is rather like Eliezer’s Outcome Pump. If you zoom out, the Outcome Pump is one part of an agent. It’s only in the zoomed in view that you see a non-sentient search process that searches for valenced empowerment over extrapolations made from running simulations of the World Model. I’d argue that something very like this planning process is occuring in AlphaZero and DeepNash (stratego). But those have narrow world models, and search systems designed to work over narrow world models.
Quote from the Outcome Pump:
So, notice this idea that humans are doing an aesthetically guided search. Seems to me this is an accurate description of human thought / planning. I think this has a lot of overlap with the aesthetic imagining of a nice picture being done by Stable Diffusion or other image models. And overlap with using Energy Models to imagine plans out of noise.