This is some evidence that it’ll work for AGIs too; after all, both humans and AGIs are massive neural nets that learn to perform diverse tasks in diverse environments.
Highly debatable whether “massive neural nets that learn to perform diverse tasks in diverse environments” is a natural category. “Massive neural net” is not a natural category—e.g. transformers vs convnets vs boltzmann machines are radically different things, to the point where understanding one tells us very little about the others. The embedding of interpretable features of one does not carry over to the others. Analytical approximations for one do not carry over to the others.
The “learn to perform diverse tasks in diverse environments” part more plausibly makes it a natural category, insofar as we buy selection theorems/conjectures.
Naturalness of categories is relative. Of course there are important differences between different kinds of massive neural nets that learn to perform diverse tasks in diverse environments. I still think it’s fair to draw a circle around all of them to distinguish them from e.g. software like Microsoft Word, or AIXI, or magic quantum suicide outcome pumps, or bacteria.
Point is that the “Structural(Inner) prediction method” doesn’t seem particularly likely to generalize across things-which-look-like-big-neural-nets. It more plausibly generalizes across things-which-learn-to-perform-diverse-tasks-in-diverse-environments, but I don’t think neural net aspect is carrying very much weight there.
Highly debatable whether “massive neural nets that learn to perform diverse tasks in diverse environments” is a natural category. “Massive neural net” is not a natural category—e.g. transformers vs convnets vs boltzmann machines are radically different things, to the point where understanding one tells us very little about the others. The embedding of interpretable features of one does not carry over to the others. Analytical approximations for one do not carry over to the others.
The “learn to perform diverse tasks in diverse environments” part more plausibly makes it a natural category, insofar as we buy selection theorems/conjectures.
Naturalness of categories is relative. Of course there are important differences between different kinds of massive neural nets that learn to perform diverse tasks in diverse environments. I still think it’s fair to draw a circle around all of them to distinguish them from e.g. software like Microsoft Word, or AIXI, or magic quantum suicide outcome pumps, or bacteria.
Point is that the “Structural(Inner) prediction method” doesn’t seem particularly likely to generalize across things-which-look-like-big-neural-nets. It more plausibly generalizes across things-which-learn-to-perform-diverse-tasks-in-diverse-environments, but I don’t think neural net aspect is carrying very much weight there.
OK, on reflection I think I tentatively agree with that.