While I see a lot of concern about the big one. I think the whole AI environment being unaligned is the more likely but not any better outcome. A society that is doing really well by some metrics that just happen to be the wrong ones. I thinking of idea of freedom of contract that was popular at the beginning of the 20th century and how hard it was to dig ourselves out of that hole.
Walker Vargas
Karma: 21
[Question] I’m looking for alternative funding strategies for cryonics.
Vigilantism has been found to be lacking. If I wanted to help with that problem in particular I’d become a cop, or vote for politicians to put higher priority on it. That seems directly comparable to what the humans in the story intended to do for most of it.
What the baby eaters are doing is worst by most people’s standards than anything in our history. At least if scale counts for something. Humans don’t even need a shared utility function. There just needs to be a cluster around what most people would reflectively endource. Paperclip maximizers might fight each other over the exact definition, but a pencil maximizer is clearly not helping by any of their standards.
Also the baby eaters aren’t Spartans. If you gave the Spartans a cure all for birth defects, they would stop killing their kids, and certainly wouldn’t complain about cure.
Do they think it’s a hardware/cost issue? Or do they think that “true” intelligence is beyond our abilities?
I think of it as deferring to future me vs. deferring to someone else.
Another consideration is how much money someone has to hand. If someone only make $1,000 a month, they may choice $25 shoes that will last a year over $100 shoes that will last 5 years. Essentially, it is the complimentary idea of economy of scale.
Personhood is a legal category and an assumed moral category that policies can point to. Usually, the rules being argued about are about the acceptability of killing something. The category is used differently depending on the moral framework, but it is usually assumed to point at the same objects. Therefore disagreements are interpreted as mistakes.
Personally, I have my doubts on there being an exact point in development that you can point to where a human becomes a person. If there is it might be weeks after birth.
If I remember right, it was in the context of there not being any universally compelling arguments. A paperclip maximizer would just ignore the tablet. It doesn’t care what the “right” thing is. Humans also probably don’t care about the cosmic tablet either. That sort of thing isn’t what “morality” is references. The argue is more of a trick to get people recognize that than a formal argument.
I think the point is that people try to point to things like God’s will in order to appear like they have a source of authority. Eliezer is trying to lead them to conclude that any such tablet being authoritative just by nature is absurd and only seems right because they expect the tablet to agree with them. Another method is asking why the tablet says what it does. Asking if God’s decrees are arbitrary or if there is a good reason, ask why not just follow those reasons.
I don’t think the fundamental ought works as a default position. Partly because there will always be a possibility of being wrong about what that fundamental ought is no matter how long it looks. So the real choice is about how sure it should be before it starts acting on it’s best known option.
The right side can’t be NULL, because that’d make the expect value of both actions NULL. To do meaningful math with these possibilities there has to be a way of comparing utilities across the scenarios.
No, if you are contributing to a preexisting discussion, there should be some older work you can cite. For example, you learned about the theory of path semantics from something that wasn’t written by you. Cite that source.
I don’t think that matrix is right. I think it describes a different scenario. Suppose an AI’s Utility function is defined referentially as being equal to some unknown function written on a letter on Mt. Everest. It also has a given utility function that it has little reason to think is correlated with the real one. Then it would be vary important to find out want that true function is. Than the expected value of any action would be NULL if that letter doesn’t exist.
But an AI that only assigns a probability that that scenario is the case might still have most of its expected value tied to following its current utility function. Well given some way of comparing them. Without that there’s no way to weigh up the choice.
I just had a thought. If Mary was presented with a red, a blue, and a green tile on a white background could she identify which was which without additional visual context clues like comparing them to her nails? If not, I would expect a p-zombie to have the same issue implying that that failure isn’t to do with consciousness.
Depending on who you are talking to for-profit corporations is a good analogy for what is meant by “misaligned”. You can then point out that those same organizations are likely to make AI with profit maximization in mind, and might skimp out on moral restraint in favor of being superhumanly good at PR.
Use that comparison with the wrong person and they’ll call you a communist.
I want to add that the AI probably does not know it is misaligned for a while.
This sounds similar to the replication crisis, in terms of the incentivization issues.
Under unification wouldn’t it make sense to consider ourselves to be every instance of our mind state? So there’s no fact of the matter of what your surroundings are like until they effect your mind state. Similarly, every past and future that is compatible with your current mind state happened and will happen respectively.
This doesn’t require faster than light signaling. If you and the copy are sent way with identical letters, that you open after crossing each other’s event horizons. You learn want was packed with your clone when you open your letter. Which lets you predict what your clone will find.
Nothing here would require the event of your clone seeing the letter to affect you. You are affected by the initial set up. If the clone counterfactually saw something else, this wouldn’t affect you according to SIA. It would require some assumptions about the setup to be wrong for that to happen to your clone though.
Another example would be if you learn a star that has crossed your cosmic event horizon was 100 solar masses, it’s fair to infer that it will become a black hole and not a white dwarf.
Sorry this is so late. I haven’t been on the site for a while. My last post was in reply to no interference always being better than fighting it out. Most of the character’s seem to think that stopping the baby eaters has more utility than letting the superhappies do the same thing to us would cost.
This is also a plausible route for spreading awareness of AI safety issues to the left. The downside is that it might make AI safety a “leftest” issue if a conservative analogy is not introduced at the same time.