Both AIXI and AIXItl will at some point drop an anvil on their own heads just to see what happens
You’re confusing arbitrary optimization with a greedy algorithm which AIXI explicitly is not. It considers a future horizon. I see you commit this falacy often. You implicitly assume “what would an arbitrarily intelligent system do?” is equivalent to “what would a arbitrarily greedy algorithm do?”
Also, the math of AIXI assumes the environment is separably divisible
I don’t know what you mean by this.
If you’re talking about the fact that it considers models of the environment where dropping the anvil on its head will yield some reward are mixed in with models of the environment where dropping an anvil on its head will result in zero reward thereafter: That’s not “assuming the environment is separably divisible”. That’s assuming the environment is seperably divisible, each program it checks is a different model for the environment as a whole. Specifically, the ones that are congruent with its experience.
Supposedly, the models that return a reward for dropping an anvil on its head would become vanishingly improbable pretty quickly, even if the system is incapable of learning about its own mortality (which it wouldn’t have to do because AIXI only works in an Agent-Environment Loop where the agent is not embedded in the environment.
If we had enough CPU time to build AIXItl, we would have enough CPU time to build other programs of similar size, and there would be things in the universe that AIXItl couldn’t model.
I think you’re missing the point of AIXI if you’re trying to think about it in practical terms. Math is a very useful human construct for studying patterns. The space of patterns describable by math is way larger than the set of patterns that exist in the real world. This creates a lot of confusion when trying to relate mathematical models to the real world.
For instance: It’s trivial to use algorithmic information theory to prove that a universal lossless compression algorithm is impossible, yet we use lossless compression to zip files all the time because we don’t live in a world that looks like TV static. Most files in the real world have a great deal of structure. Most N-length bit-strings of the set of all possible N-length bitstrings have no discernable structure at all.
I think it’s easy to get distracted by the absurdity of AIXI and miss the insight that it provides.
AIXItl (but not AIXI, I think) contains a magical part: namely a theorem-prover which shows that policies never promise more than they deliver.
What does that even mean? How does AIXItl promise something?
My main problem is that I don’t think Hutter should have titled his book “Universal Artificial Intelligence”. For one because I don’t think the word “artificial belongs”, but mainly because I don’t think optimality is equivalent to intelligence nor do I think intelligence should be defined in terms of an agent-environment loop. That implies that a system with less agency, like Stephen Hawking, would be less intelligent.
I think of intelligence as a measure of a system’s ability to produce solutions to problems. Unlike optimality, I think the resources required to produce a given solution should also factor in. That ends up being highly circumstancial, so it ends up being a relative measure rather than something absolute. AIXIlt being as brute-force as an algorithm gets, would probably compare poorly to most heuristic systems.
You’re confusing arbitrary optimization with a greedy algorithm which AIXI explicitly is not. It considers a future horizon. I see you commit this falacy often. You implicitly assume “what would an arbitrarily intelligent system do?” is equivalent to “what would a arbitrarily greedy algorithm do?”
I don’t know what you mean by this.
If you’re talking about the fact that it considers models of the environment where dropping the anvil on its head will yield some reward are mixed in with models of the environment where dropping an anvil on its head will result in zero reward thereafter: That’s not “assuming the environment is separably divisible”. That’s assuming the environment is seperably divisible, each program it checks is a different model for the environment as a whole. Specifically, the ones that are congruent with its experience.
Supposedly, the models that return a reward for dropping an anvil on its head would become vanishingly improbable pretty quickly, even if the system is incapable of learning about its own mortality (which it wouldn’t have to do because AIXI only works in an Agent-Environment Loop where the agent is not embedded in the environment.
I think you’re missing the point of AIXI if you’re trying to think about it in practical terms. Math is a very useful human construct for studying patterns. The space of patterns describable by math is way larger than the set of patterns that exist in the real world. This creates a lot of confusion when trying to relate mathematical models to the real world.
For instance: It’s trivial to use algorithmic information theory to prove that a universal lossless compression algorithm is impossible, yet we use lossless compression to zip files all the time because we don’t live in a world that looks like TV static. Most files in the real world have a great deal of structure. Most N-length bit-strings of the set of all possible N-length bitstrings have no discernable structure at all.
I think it’s easy to get distracted by the absurdity of AIXI and miss the insight that it provides.
What does that even mean? How does AIXItl promise something?
My main problem is that I don’t think Hutter should have titled his book “Universal Artificial Intelligence”. For one because I don’t think the word “artificial belongs”, but mainly because I don’t think optimality is equivalent to intelligence nor do I think intelligence should be defined in terms of an agent-environment loop. That implies that a system with less agency, like Stephen Hawking, would be less intelligent.
I think of intelligence as a measure of a system’s ability to produce solutions to problems. Unlike optimality, I think the resources required to produce a given solution should also factor in. That ends up being highly circumstancial, so it ends up being a relative measure rather than something absolute. AIXIlt being as brute-force as an algorithm gets, would probably compare poorly to most heuristic systems.