This rather reminds be of a discussion (which alas I cannot recall a reference for) of the idea that a system having “free will” was an illusion (or at least a heuristic viewpoint) induced by computational resource limitations of not having sufficient computational resources and data to fully model and predict the system’s behavior, and that thus something that to us looks like it has free will (including ourselves) might be seen as entirely deterministic or probabilistic to an agent with much higher computational and data resources. If you can predict the behavior of a computationally-bounded (and thus not fully rational) agent sufficiently well to sucessfully Dutch-book them, because you have significantly more computational resources than them, then they probably don’t look to you as if they entirely have “free will”.
Yes! I’m a fan of Yudkowsky’s view that the sensation of free will is the sensation of “couldness” among multiple actions. When it feels like I could do one thing or another, it feels like I have free will. When it feels like I could have chosen differently, it feels like I chose freely.
I suspect that an important ingredient of the One True Decision Theory is being shaped in such a way that other agents, modelling how you’ll respond to different policies they might implement, find it in their interest to implement policies which treat you fairly.
This rather reminds be of a discussion (which alas I cannot recall a reference for) of the idea that a system having “free will” was an illusion (or at least a heuristic viewpoint) induced by computational resource limitations of not having sufficient computational resources and data to fully model and predict the system’s behavior, and that thus something that to us looks like it has free will (including ourselves) might be seen as entirely deterministic or probabilistic to an agent with much higher computational and data resources. If you can predict the behavior of a computationally-bounded (and thus not fully rational) agent sufficiently well to sucessfully Dutch-book them, because you have significantly more computational resources than them, then they probably don’t look to you as if they entirely have “free will”.
Yes! I’m a fan of Yudkowsky’s view that the sensation of free will is the sensation of “couldness” among multiple actions. When it feels like I could do one thing or another, it feels like I have free will. When it feels like I could have chosen differently, it feels like I chose freely.
I suspect that an important ingredient of the One True Decision Theory is being shaped in such a way that other agents, modelling how you’ll respond to different policies they might implement, find it in their interest to implement policies which treat you fairly.