General performance in some domain requires learning the entire corresponding abstraction layer, but if a model’s performance is evaluated only on some narrow task within that domain, it’ll just overfit to that task.
I think would be better to say “overkill” instead of “overfit” at the end of this sentence because overfit is technically the opposite of learning universal, highly regularised models.
The latter is the commonality across the models, the redundant information we’re looking for: their ability to win. The noisy environment of the fluctuating rules would wipe out any details about the heuristics they use, leaving only the signal of “this agent performs well”. The high-level abstraction, then, would be “something that wins given a ruleset”. Something that outputs actions that lead to low loss no matter the environment it’s in. Something that, given some actions it can take, always picks those that lead to low loss because they lead to low loss.
Consequentialism. Agency. An optimizer.
As well as here, I think it would be clearer to skip the word “agency” and just say “A consequentialistic optimiser”.
Chemistry, biology, psychology, geopolitics, cosmology — it’s all downstream of fundamental physics, yet the objects at any level behave very unlike the objects at a different level.
All mentioned (scientific) theories are not reducible to the time-symmetric evolution of the universe as a quantum system, which scientists currently think the universe is. That would be reductionism. They are also all classical and time-asymmetric, which means they are semantics on top of quantum physical systems.
And if you’re teaching yourself to play by completely novel rules, how can you even tell whether you’re performing well, without the inner notion of a goal to pursue?
Saying “play by rules” implies there is some game with the rules of winning and losing (though, a game may not have winners and losers, too). So, either I can tell I’m playing well if the rules of winning and losing are given to me along with the rest of the rules, or I’m seeking pure fun (see open-endedness, also formalised as “seeking surprise” in Active Inference). Fun (surprise, interestingness) can be formalised, though it doesn’t need to be equal to “performing well”.
I think would be better to say “overkill” instead of “overfit” at the end of this sentence because overfit is technically the opposite of learning universal, highly regularised models.
As well as here, I think it would be clearer to skip the word “agency” and just say “A consequentialistic optimiser”.
All mentioned (scientific) theories are not reducible to the time-symmetric evolution of the universe as a quantum system, which scientists currently think the universe is. That would be reductionism. They are also all classical and time-asymmetric, which means they are semantics on top of quantum physical systems.
Saying “play by rules” implies there is some game with the rules of winning and losing (though, a game may not have winners and losers, too). So, either I can tell I’m playing well if the rules of winning and losing are given to me along with the rest of the rules, or I’m seeking pure fun (see open-endedness, also formalised as “seeking surprise” in Active Inference). Fun (surprise, interestingness) can be formalised, though it doesn’t need to be equal to “performing well”.