Robin: I think Eliezer’s question is worth thinking about now. If you do investigate what you would wish from a genie, isn’t it possible that one of your wishes might be easy enough for you to grant without the genie? You do say you haven’t thought about the question yet, so you really have no way of knowing whether your wishes would actually be that difficult to grant.
Questions like “what do I want out of life?” or “what do I want the universe to look like?” are super important questions to ask, regardless of whether you have a magic genie. I personally have had the unfortunate experience of answering some parts of those question wrong and then using those wrong answers to run my life for a while. All I have to say on the matter is that that situation is definitely worth avoiding. I still don’t expect my present set of answers to be right. I think they’re marginally more right than they were three years ago.
You don’t have a genie, but you do have a human brain, which is a rather powerful optimization process and despite not being a genie it is still very capable of shooting its owner in the foot. You should check what you think you want in the limiting case of absolute power, because if that’s not what you want, then you got it wrong. If you think the meaning of life is to move westward, but then you think about the actual lay of the land hundreds of miles west of where you are and then discover you wouldn’t like that, then it’s worth trying to more carefully formulate why it was you wanted to go west in the first place, and once you know the reason, maybe going north is even better. If you don’t want to waste time moving in the wrong direction then it’s important to know what you want as clearly as possible.
Robin: I think Eliezer’s question is worth thinking about now. If you do investigate what you would wish from a genie, isn’t it possible that one of your wishes might be easy enough for you to grant without the genie? You do say you haven’t thought about the question yet, so you really have no way of knowing whether your wishes would actually be that difficult to grant.
Questions like “what do I want out of life?” or “what do I want the universe to look like?” are super important questions to ask, regardless of whether you have a magic genie. I personally have had the unfortunate experience of answering some parts of those question wrong and then using those wrong answers to run my life for a while. All I have to say on the matter is that that situation is definitely worth avoiding. I still don’t expect my present set of answers to be right. I think they’re marginally more right than they were three years ago.
You don’t have a genie, but you do have a human brain, which is a rather powerful optimization process and despite not being a genie it is still very capable of shooting its owner in the foot. You should check what you think you want in the limiting case of absolute power, because if that’s not what you want, then you got it wrong. If you think the meaning of life is to move westward, but then you think about the actual lay of the land hundreds of miles west of where you are and then discover you wouldn’t like that, then it’s worth trying to more carefully formulate why it was you wanted to go west in the first place, and once you know the reason, maybe going north is even better. If you don’t want to waste time moving in the wrong direction then it’s important to know what you want as clearly as possible.