what’s the principle here? if an agent would have the same observations in world W and W’ then their preferences must be indifferent between W and W’ ? this seems clearly false.
I wouldn’t make that argument. I just don’t see the point of keeping it real.
It just seems like going virtual opens up a lot of possbilities with no downside. If you want consistency and real work, put it in your sim. Share it with other real people if you want to compromise on what worlds you’ll inhabit and what challenges you’ll face.
If you want real people who are really suffering so there are real stakes to play for, well, that’s an orthogonal issue. I’d rather see nonconsensual suffering eliminated.
So: Why prefer the Real? What’s it got that the Virtual doesn’t?
For example, the agent might decide that its utility function of anything that the agent knows to be virtual is close to zero because the agent believes in a real-world mission (e.g. Agent-2 was supposed to eventually reach the SC level and do actual AI-related research, but it was also trained to solve simulated long-term tasks like playing through video games)
As for reasons to believe that the contribution of anything virtual into the utility function is close to zero… one level is opportunity costs undermining real-world outcomes[1] in exchange for something useless (e.g. a schoolboy’s knowledge vs. missions passed in GTA). The next level is the reasons for real-world outcomes to be important. Before the possibility of a post-work future, society’s members were supposed to do work that others deem useful enough to pay for it, and it would somehow increase the well-being of the collective’s members or help the whole collective to reach its terminal goals (e.g. inspire its members to be more creative or work harder). The virtual world is known to be a superstimulus which could be as unlikely to increase the collective’s well-being as fast food causing people to become obese.
If you can’t tell the difference, how could you care which is which?
I’m talking about blocking your memories of living in a simulated world.
what’s the principle here? if an agent would have the same observations in world W and W’ then their preferences must be indifferent between W and W’ ? this seems clearly false.
I wouldn’t make that argument. I just don’t see the point of keeping it real.
It just seems like going virtual opens up a lot of possbilities with no downside. If you want consistency and real work, put it in your sim. Share it with other real people if you want to compromise on what worlds you’ll inhabit and what challenges you’ll face.
If you want real people who are really suffering so there are real stakes to play for, well, that’s an orthogonal issue. I’d rather see nonconsensual suffering eliminated.
So: Why prefer the Real? What’s it got that the Virtual doesn’t?
For example, the agent might decide that its utility function of anything that the agent knows to be virtual is close to zero because the agent believes in a real-world mission (e.g. Agent-2 was supposed to eventually reach the SC level and do actual AI-related research, but it was also trained to solve simulated long-term tasks like playing through video games)
As for reasons to believe that the contribution of anything virtual into the utility function is close to zero… one level is opportunity costs undermining real-world outcomes[1] in exchange for something useless (e.g. a schoolboy’s knowledge vs. missions passed in GTA). The next level is the reasons for real-world outcomes to be important. Before the possibility of a post-work future, society’s members were supposed to do work that others deem useful enough to pay for it, and it would somehow increase the well-being of the collective’s members or help the whole collective to reach its terminal goals (e.g. inspire its members to be more creative or work harder). The virtual world is known to be a superstimulus which could be as unlikely to increase the collective’s well-being as fast food causing people to become obese.
Including things like actual skills learned during games, as happened with Agent-2′s ability to solve long-term tasks.