This will make more sense if you have a basic grasp on quantum mechanics, but if you’re willing to accept “energy comes in discrete units” as a premise then you should be mostly fine.
My current understanding is that QM is not-at-all needed to make sense of stat mech. Instead, the thing where energy is equally likely to be in any of the degrees of freedom just comes from using a measure over your phase space such that the dynamical law of your system preservers that measure!
Hm… so anything that measures degree of agent structure should register a policy with a sub-agent as having some agent structure. But yeah, I haven’t thought much about the scenarios where there are multiple agents inside the policy. The agent structure problem is trying to use performance to find a minimum measure of agent structure. So if there was an agent hiding in there that didn’t impact the performance during the measured time interval, then it wouldn’t be detected (although it would detect it “in the limit”).
That said, we’re not actually talking about how to measure degree of agent structure yet. It seems plausible to me that whatever method one uses to do that could be adapted to find multiple agents.