(For example, if you expect that an external actuator is watching your beliefs and trying to make them false, then you’re stuck at 50% unless you can fool them or yourself.) But we don’t live in an arbitrarily adversarial environment. Indeed, from a Knightian perspective a big part of rationality is figuring out which parts of your environment are friendly, and which are adversarial, and steering towards the former.
if the action space is not Boolean, it’s much worse than 50%, the pessimizer just has to select any random action out of the millions possible ones (some concept of “closeness” would be still needed to think about it clearly, e.g. to cause a mis-step the adversarial “leg agent” should mess with fine muscle movements and not start growing a cancer)
not a big deal if steering/moving in concept space and causing/creating the friendly environment are equal in the formalism you have in mind, but if the difference between discovery and invention is large enough for a given agent, viable strategies for selecting friendly environments might be over-determined by affordances rather than raw intelligence
🤔 could it be conceptually saved by modelling it as expected energy that can be used for bounded rationality computations vs other actions interchangeably, conceptually replacing “probability” as the universal currency? or maybe as multiple non-fungible forces while at human level (we are not smart enough to be ideally rational on single dimension so we optimize for multiple constraints with uncertainty/non-linearities about the trade-offs between survival, expected utility, sounds-too-sophisticated-to-not-make-a-mistake heuristic, …) vs single force a la electro-weak in physics for high energies/intelligence… in any case I notice I am confused about the implications, but I feel the importance of “arbitrary adversarial environment” looks under-considered for further development of the Knightian quantification thingy
e.g. it might be better to let our non-evil political enemies win a fight that to burn the bridges and poison the wells, but if I or some egregore is about to wither and die, then obviously the enemies are evil and we should turn the world into chaos for a chance of hope, however slim the odds
if the action space is not Boolean, it’s much worse than 50%, the pessimizer just has to select any random action out of the millions possible ones (some concept of “closeness” would be still needed to think about it clearly, e.g. to cause a mis-step the adversarial “leg agent” should mess with fine muscle movements and not start growing a cancer)
not a big deal if steering/moving in concept space and causing/creating the friendly environment are equal in the formalism you have in mind, but if the difference between discovery and invention is large enough for a given agent, viable strategies for selecting friendly environments might be over-determined by affordances rather than raw intelligence
🤔 could it be conceptually saved by modelling it as expected energy that can be used for bounded rationality computations vs other actions interchangeably, conceptually replacing “probability” as the universal currency? or maybe as multiple non-fungible forces while at human level (we are not smart enough to be ideally rational on single dimension so we optimize for multiple constraints with uncertainty/non-linearities about the trade-offs between survival, expected utility, sounds-too-sophisticated-to-not-make-a-mistake heuristic, …) vs single force a la electro-weak in physics for high energies/intelligence… in any case I notice I am confused about the implications, but I feel the importance of “arbitrary adversarial environment” looks under-considered for further development of the Knightian quantification thingy
e.g. it might be better to let our non-evil political enemies win a fight that to burn the bridges and poison the wells, but if I or some egregore is about to wither and die, then obviously the enemies are evil and we should turn the world into chaos for a chance of hope, however slim the odds