A bit has two possible values, 0 and 1, it doesn’t have “destroyed” value. And, of course, it’s not enough to prefer intact laptop over one wholly pulverised into plasma (the latter severs the cable connecting the laptop to the internet, after all). There’s necessarily some parts of the laptop which in reality are necessary to compute AIXI but which in AIXI’s model do not compute the output actions of AIXI (as potential actions are magic’d into the model rather than arise through processes in the model). Those parts are liable to being tampered with, especially as any changes in their behaviour would be grossly misunderstood (e.g. increase in clock speed would be misunderstood as world slowing down).
edit: And especially as existence of those parts is detrimental to the operation of the protected part of the laptop (due to their power consumption, heat, risk of short circuit failure taking out the protected part, etc). Somewhat simplifying, in reality there’s the CPU internals that compute AIXI, and there’s the bus controller that sends the actions onto the bus, to eventually act on the real world, and that reads the rewards. In AIXI’s model of self, there’s useless hardware (deep internals whose output is substituted for with potential actions) that is connected to the same power supply as the critically important components (that relay the actions and rewards), endangering their operation.
A bit has two possible values, 0 and 1, it doesn’t have “destroyed” value.
A physical bit does. Remember that we are talking about an actual bit stored inside a memory location on the computer (say, a capacitor in a DRAM cell).
And, of course, it’s not enough to prefer intact laptop over one wholly pulverised into plasma
Why not? Not recieving any future reward is such a huge negative utility that it would take a very large positive utility to carry out an action that would risk that occuring. Would you allow a surgeon to remove some section of your brain for $1,000,000 even if you knew that that section would not affect your reward pathways?
Would you allow a surgeon to remove some section of your brain for $1,000,000 even if you knew that that section would not affect your reward pathways?
If I had brain cancer or cerebral AVM or the like, I’d pay to have it removed. See my edit. The root issue is that in AIXI’s model, potential actions (that it iterates through) are not represented as output of some hardware, but are forced onto the model. Consequently the hardware that actually outputs those in the real world is not represented as critical. And it is connected in parallel onto the same power supply (as the understood-to-be-critically-important hardware which relays the actions). It literally thinks it got a brain parasite. Of course it won’t necessarily drop an anvil at the whole thing just because of experimenting—that’s patently stupid. It will surgically excise some parts with great caution.
A bit has two possible values, 0 and 1, it doesn’t have “destroyed” value. And, of course, it’s not enough to prefer intact laptop over one wholly pulverised into plasma (the latter severs the cable connecting the laptop to the internet, after all). There’s necessarily some parts of the laptop which in reality are necessary to compute AIXI but which in AIXI’s model do not compute the output actions of AIXI (as potential actions are magic’d into the model rather than arise through processes in the model). Those parts are liable to being tampered with, especially as any changes in their behaviour would be grossly misunderstood (e.g. increase in clock speed would be misunderstood as world slowing down).
edit: And especially as existence of those parts is detrimental to the operation of the protected part of the laptop (due to their power consumption, heat, risk of short circuit failure taking out the protected part, etc). Somewhat simplifying, in reality there’s the CPU internals that compute AIXI, and there’s the bus controller that sends the actions onto the bus, to eventually act on the real world, and that reads the rewards. In AIXI’s model of self, there’s useless hardware (deep internals whose output is substituted for with potential actions) that is connected to the same power supply as the critically important components (that relay the actions and rewards), endangering their operation.
A physical bit does. Remember that we are talking about an actual bit stored inside a memory location on the computer (say, a capacitor in a DRAM cell).
Why not? Not recieving any future reward is such a huge negative utility that it would take a very large positive utility to carry out an action that would risk that occuring. Would you allow a surgeon to remove some section of your brain for $1,000,000 even if you knew that that section would not affect your reward pathways?
If I had brain cancer or cerebral AVM or the like, I’d pay to have it removed. See my edit. The root issue is that in AIXI’s model, potential actions (that it iterates through) are not represented as output of some hardware, but are forced onto the model. Consequently the hardware that actually outputs those in the real world is not represented as critical. And it is connected in parallel onto the same power supply (as the understood-to-be-critically-important hardware which relays the actions). It literally thinks it got a brain parasite. Of course it won’t necessarily drop an anvil at the whole thing just because of experimenting—that’s patently stupid. It will surgically excise some parts with great caution.