CEV is ~ optimizing the world based on what you would later wish, not just warning you (though the optimization could include just warning you about some things). Good start though!
I currently think it’s a better operationalization of CEV to not “optimize based on what you’d later want”, exactly.
People frequently object “but future me want all kinds of path dependent alien stuff”
To which I’ve replied “but, it only does the stuff that lots of different monte carlo simulations all turn out to want”
To which people “I dunno still seems like the me who’s thought for 1000 years or whatever may end up alien in some way, and I don’t sign up to automatically identify with that.”
In my last discussion about at this, I said “I think the right way to do CEV is, you don’t optimize the values of the thing-at-the-end. You optimize what future-you would do if they were specifically trying to help present you (or, by present you’s values after having some kind of mediated conversation with various chains of future you’s). And then, only where the values cohere across time and simulation-rolls.
(might easily change my mind about this. I’m not sure if there’s more context I’m missing)
“Whenever you’d certainly later wish you’d been warned against your course of action, you are.”
CEV is ~ optimizing the world based on what you would later wish, not just warning you (though the optimization could include just warning you about some things). Good start though!
I currently think it’s a better operationalization of CEV to not “optimize based on what you’d later want”, exactly.
People frequently object “but future me want all kinds of path dependent alien stuff”
To which I’ve replied “but, it only does the stuff that lots of different monte carlo simulations all turn out to want”
To which people “I dunno still seems like the me who’s thought for 1000 years or whatever may end up alien in some way, and I don’t sign up to automatically identify with that.”
In my last discussion about at this, I said “I think the right way to do CEV is, you don’t optimize the values of the thing-at-the-end. You optimize what future-you would do if they were specifically trying to help present you (or, by present you’s values after having some kind of mediated conversation with various chains of future you’s). And then, only where the values cohere across time and simulation-rolls.
(might easily change my mind about this. I’m not sure if there’s more context I’m missing)