if I can learn to predict that my perceptual stream will end if I drop an anvil on my head even though I’ve never tried it, why can’t an AIXI agent?
Predicting anvils isn’t enough, you also need the urge to avoid them. You got that urge from evolution, which got it by putting many creatures in dangerous situations and keeping those that survived. I’m not sure if AIXI can learn that without having any experience with death.
Predicting anvils isn’t enough, you also need the urge to avoid them. You got that urge from evolution, which got it by putting many creatures in dangerous situations and keeping those that survived. I’m not sure if AIXI can learn that without having any experience with death.