Ah man, I’ve been missing these sorts of posts from you, very happy to read this, super cool as usual.
These are some questions that arise for me, maybe not the most well out but hopefully somewhat interesting:
Do you think this is a good argument for multi-scale modularity in biology? Also thoughts on multi-agent models of mind with this model in the background? Finally any thoughts on applying this to whether we will have a group of AI systems doing RSI or a singular one?
I suppose this argument would be dependent on the condition when systems with convex utility functions show up, do you have any more detailed thoughts on when we can expect convex utility trade offs to show up?
Is the answer something like when there are benefits to specialization? I guess there also has to be the aspect of trade already present in the system? Do you have thoughts on the assumptions you have to make and have true before the convergence arises or do you think this is a general property of any learning system?
Do you think this is a good argument for multi-scale modularity in biology?
I think the multi-scale aspect is mostly orthogonal to this argument, but yeah the argument could be applied at multiple scales.
Also thoughts on multi-agent models of mind with this model in the background? Finally any thoughts on applying this to whether we will have a group of AI systems doing RSI or a singular one?
Typically, the load-bearing aspect of multi-agent models is that the multiple agents have different utility functions. This argument is compatible with multiple utility functions in principle, but that’s not really what it’s about; it would apply arguably-more-cleanly in systems with just one optimization objective (but multiple subgoals, e.g. multiple resources like apples and bananas).
I guess there also has to be the aspect of trade already present in the system?
Right so I was trying to make sense of this no trade thing because I thought it would make it so that the individual subcomponents would die out if they didn’t have access to trade but that isn’t the case if they’re subordinated to the larger system. Also similar to the multi-agent thing it’s not about multi-scale modularity but rather optimisation within a specific optimisation system or whatever you would like to call it. (random quick picture):
Ah man, I’ve been missing these sorts of posts from you, very happy to read this, super cool as usual.
These are some questions that arise for me, maybe not the most well out but hopefully somewhat interesting:
Do you think this is a good argument for multi-scale modularity in biology? Also thoughts on multi-agent models of mind with this model in the background? Finally any thoughts on applying this to whether we will have a group of AI systems doing RSI or a singular one?
I suppose this argument would be dependent on the condition when systems with convex utility functions show up, do you have any more detailed thoughts on when we can expect convex utility trade offs to show up?
Is the answer something like when there are benefits to specialization? I guess there also has to be the aspect of trade already present in the system? Do you have thoughts on the assumptions you have to make and have true before the convergence arises or do you think this is a general property of any learning system?
I think the multi-scale aspect is mostly orthogonal to this argument, but yeah the argument could be applied at multiple scales.
Typically, the load-bearing aspect of multi-agent models is that the multiple agents have different utility functions. This argument is compatible with multiple utility functions in principle, but that’s not really what it’s about; it would apply arguably-more-cleanly in systems with just one optimization objective (but multiple subgoals, e.g. multiple resources like apples and bananas).
Nope.
(The other questions I don’t yet have much to say about.)
Right so I was trying to make sense of this no trade thing because I thought it would make it so that the individual subcomponents would die out if they didn’t have access to trade but that isn’t the case if they’re subordinated to the larger system. Also similar to the multi-agent thing it’s not about multi-scale modularity but rather optimisation within a specific optimisation system or whatever you would like to call it. (random quick picture):