I wish I could upvote this more than once. An agent not yet capable of 100% precommitments can’t be expected to make them, but you can imagine it slowly self-modifying to install stronger and stronger internal pre-commitment mechanisms, since even probabilistic precommitments are better than nothing (like Joe at least tried to do in Newcomb’s problem happened to me).
Then it would fare more optimally on average for scenarios that involve predicting its behavior after those installments. I think that’s the most one can hope for if one does not start with precommitment mechanisms. You can’t stop Omega (in Judea Pearl’s timeless sense of causality, not classically) from simulating a bad decision you would have made one year ago, but you can stop him from simulating a bad decision you would make in the future.
I wish I could upvote this more than once. An agent not yet capable of 100% precommitments can’t be expected to make them, but you can imagine it slowly self-modifying to install stronger and stronger internal pre-commitment mechanisms, since even probabilistic precommitments are better than nothing (like Joe at least tried to do in Newcomb’s problem happened to me).
Then it would fare more optimally on average for scenarios that involve predicting its behavior after those installments. I think that’s the most one can hope for if one does not start with precommitment mechanisms. You can’t stop Omega (in Judea Pearl’s timeless sense of causality, not classically) from simulating a bad decision you would have made one year ago, but you can stop him from simulating a bad decision you would make in the future.
I’ll take it :)