I like this analysis. Whereas Seeking Power is Instrumentally Convergent in MDPs aimed to explain why seeking power is instrumentally convergent, this predicts qualitative properties of resource-hungry policies (“only cares about resource-relevant summary f(X) to the extent it affects long-term action spaces”). I think this frame also makes plain that most sensory reward functions don’t differentiate between “humans flourish + I have power” world histories, and “humans perish + I have power” world histories.
More generally, I think mediating-long-term-action-space is part of how we intuitively decide what to call “resources” in the first place.
At this point, I ground my understanding of resources in the POWER(s) formulation (ability to achieve goals in general), and I think your take agrees with my model of that. Most goals we could have care more about the long-term action space, and so care about the stable and abstraction-friendly components of the environment which can widen that space (eg “resources”). Alternatively, resources are those things which tend to increase your POWER.
I like this analysis. Whereas Seeking Power is Instrumentally Convergent in MDPs aimed to explain why seeking power is instrumentally convergent, this predicts qualitative properties of resource-hungry policies (“only cares about resource-relevant summary f(X) to the extent it affects long-term action spaces”). I think this frame also makes plain that most sensory reward functions don’t differentiate between “humans flourish + I have power” world histories, and “humans perish + I have power” world histories.
At this point, I ground my understanding of resources in the POWER(s) formulation (ability to achieve goals in general), and I think your take agrees with my model of that. Most goals we could have care more about the long-term action space, and so care about the stable and abstraction-friendly components of the environment which can widen that space (eg “resources”). Alternatively, resources are those things which tend to increase your POWER.