In your article, you seemed to be saying that you specifically think that one shouldn’t have a single “final decision” function at the top of the meta stack. That’s not going to be an easily accepted argument around here, for the reasons I stated above.
Yeah, this is exactly what I am arguing.
For technical reasons, it is always possible (and also usually helpful) to describe this as a single function or algorithm, typically around here called one’s “utility function” or “terminal values”.
Could you explain the technical reasons more, or point me to some essays where I could read about this? I am still not convinced why it is more benefical to have a single operating system.
I’m no technical expert, but: if I want X, and I also want Y, and I also want Z, and I also want W, and I also want A1 through A22, it seems pretty clear to me that I can express those wants as “I want X and Z and W and A1 through A22.” Talking about whether I have one goal or 26 goals therefore seems like a distraction.
In regards to why it’s possible, I’ll just echo what TheOtherDaveSaid.
The reason it’s helpful to try for a single top-level utility function is because otherwise, whenever there’s a conflict among the many many things we value, we’d have no good way to consistently resolve it. If one aspect of your mind wants excitement, and another wants security, what should you do when you have to choose between the two?
Is quitting your job a good idea or not? Is going rock climbing instead of staying at home reading this weekend a good idea or not? Different parts of your mind will have different opinions on these subjects. Without a final arbiter to weigh their suggestions and consider how important comfort and security are relative to each other, how do you do decide in a non-arbitrary way?
So I guess it comes down to: how important is it to you that your values are self-consistent?
More discussion (and a lot of controversy on whether the whole notion actually is a good idea) here.
Without a final arbiter to weigh their suggestions and consider how important comfort and security are relative to each other, how do you do decide in a non-arbitrary way?
Well, there’s always the approach of letting all of me influence my actions and seeing what I do.
Yeah, this is exactly what I am arguing.
Could you explain the technical reasons more, or point me to some essays where I could read about this? I am still not convinced why it is more benefical to have a single operating system.
I’m no technical expert, but: if I want X, and I also want Y, and I also want Z, and I also want W, and I also want A1 through A22, it seems pretty clear to me that I can express those wants as “I want X and Z and W and A1 through A22.” Talking about whether I have one goal or 26 goals therefore seems like a distraction.
In regards to why it’s possible, I’ll just echo what TheOtherDaveSaid.
The reason it’s helpful to try for a single top-level utility function is because otherwise, whenever there’s a conflict among the many many things we value, we’d have no good way to consistently resolve it. If one aspect of your mind wants excitement, and another wants security, what should you do when you have to choose between the two?
Is quitting your job a good idea or not? Is going rock climbing instead of staying at home reading this weekend a good idea or not? Different parts of your mind will have different opinions on these subjects. Without a final arbiter to weigh their suggestions and consider how important comfort and security are relative to each other, how do you do decide in a non-arbitrary way?
So I guess it comes down to: how important is it to you that your values are self-consistent?
More discussion (and a lot of controversy on whether the whole notion actually is a good idea) here.
Well, there’s always the approach of letting all of me influence my actions and seeing what I do.
Thanks for the link. I’ll respond back when I get a chance to read it.