tailcalled

• tailcalled 26 Mar 2015 16:58 UTC

  5 points

  on: Values at com­pile time

  Problem one: It would require the human to be able to correctly design utopia (or at least not a dystopia—being able to design a not-dystopia is probably rarer than one might think).

  Problem two: There are moral problems in letting an AI simulate a human in sufficiently high detail.

  Problem three: In certain cases, the human might command things that they do not want. In particular, if the human-module is simulated as essentially a human within the AI, the AI might wirehead the human and ask if humans in general should be wireheaded (or something like that).

• tailcalled 26 Mar 2015 17:12 UTC

  5 points

  on: Re­vis­it­ing Non-centrality

  So what you’re basically proposing is that instead of saying “NONCENTRAL FALLACY!!1!!1!”, people simply don’t mention the fallacy and instead compare the possibly-noncentral element to the typical element of the group and explain why it’s different?

  Because if so, I would agree, but I don’t think it’s unique to the noncentral fallacy. In general, if someone uses a fallacious argument, you should be able to directly show the problem, without mentioning the fallacy by name.

  … however, what you are doing is still, essentially, saying that he ‘is the good kind of criminal’, just in a more complicated way that the people you are debating with hopefully won’t notice.

• tailcalled 27 Mar 2015 13:29 UTC

  2 points

  in reply to: TheAncientGeek’s comment on: Values at com­pile time

  1. Most people I’ve talked to have one or two world changing schemes that they want to implement. This might be selection bias, though.

  2. It is not at all obvious to me that any optimizer would be personlike. Sure, it would be possible (maybe even easy!) to build a personlike AI, but I’m not sure it would “necessarily” happen. So I don’t know if those problems would be there for an arbitrary AI, but I do know that they would be there for its models of humans.

• tailcalled 27 Mar 2015 13:37 UTC

  2 points

  in reply to: Stuart_Armstrong’s comment on: Values at com­pile time

  1. Which leads to the obvious question of whether figuring out the rules about the questions is much simpler than figuring out the rules for morality. Do you have a specific, simple class of questions/​orders in mind?

  2. Yes, but it seems to me that your approach is dependent on an ‘immoral’ system: simulating humans in too high detail. In other cases, one might attempt to make a nonperson predicate and eliminate all models that fail, or something. However, your idea seems to depend on simulated humans.

  3. Well, it depends on how the model of the human works and how it is asked questions. That would probably depend a lot on how the original AI structured the model of the human, and we don’t currently have any AIs to test that with. The point is, though, that in certain cases, the AI might compromise the human, for instance by wireheading it or convincing it of a religion or something, and then the compromised human might command destructive things. There’s a huge, hidden amount of trickiness, such as determining how to give the human correct information to decide etc etc.

Box­ing an AI?

• tailcalled 27 Mar 2015 14:15 UTC

  0 points

  in reply to: Stuart_Armstrong’s comment on: Values at com­pile time

  The problem is that the ‘human interpretation module’ might give the wrong results. For instance, if it convinces people that X is morally obligatory, it might interpret that as X being morally obligatory. It is not entirely obvious to me that it would be useful to have a better model. It probably depends on what the original AI wants to do.

• tailcalled 27 Mar 2015 14:19 UTC

  0 points

  in reply to: casebash’s comment on: Re­vis­it­ing Non-centrality

  My point is, however, that this is not unique to noncentral fallacy.

• tailcalled 27 Mar 2015 14:25 UTC

  2 points

  in reply to: TheAncientGeek’s comment on: Values at com­pile time

  I dislike the concept of qualia because it seems to me that it’s just a confusing name for “how inputs feel from the inside of an algorithm”.

• tailcalled 27 Mar 2015 15:54 UTC

  1 point

  in reply to: gjm’s comment on: Box­ing an AI?

  I’m not saying it would solve everything, I’m saying it would be a way to test significant aspects of AI without destroying the world, including significant aspects of their morality. It’s not a “do this magic and morality for AI is solved” as much as a “this doable step helps parts of AI design, probably including preventing the worst classes of paperclip-maximization”.

• tailcalled 27 Mar 2015 15:55 UTC

  4 points

  in reply to: AABoyles’s comment on: Box­ing an AI?

  If the AI is sufficiently intelligent, it should be able to observe that its computational resources are bounded, and infer the existence of the box.

  I don’t see how ‘box’ follows from ‘bounded computational resources’. Could you explain the logic?

• tailcalled 27 Mar 2015 15:57 UTC

  1 point

  in reply to: Error’s comment on: Box­ing an AI?

  If I remember right, this has already been considered and the argument against it is that any AI powerful enough to be interesting will also have a chance to correctly guess that it’s in a box, for more or less the same reason that you or I can come up with the Simulation Hypothesis.

  Well, yes, it will probably come up with the hypothesis, but it has no evidence for it and even if it had, it does not have enough information about how we work to be able to manipulate us.

• tailcalled 27 Mar 2015 16:35 UTC

  2 points

  in reply to: Stuart_Armstrong’s comment on: Values at com­pile time

  I know, but my point is that such a model might be very perverse, such as “Humans do not expect to find out that you presented misleading information.” rather than “Humans do not expect that you present misleading information.”

• tailcalled 27 Mar 2015 16:36 UTC

  0 points

  in reply to: chaosmage’s comment on: Box­ing an AI?

  … how would the AI find out that it is in a box if it isn’t told about it?

• tailcalled 27 Mar 2015 17:35 UTC

  −1 points

  in reply to: chaosmage’s comment on: Box­ing an AI?

  How would you find out something that a three-year-old is trying to hide from you?

  A three-year-old that designed the universe I live in in such a way that it stays hidden as well as possible? I have absolutely no idea. I would probably hope for it to get bored and tell me.

  We don’t have any way of knowing whether a deception that would fool the likes of us would fool somebody way smarter.

  We do have at least some clues:

  1. We know the mathematics of how optimal belief updating works.

  2. We have some rough estimates of the complexity of the theory the AI must figure out.

  3. We have some rough estimates of the amount of information we have given the AI.

  I know there is such a thing as underestimating AI, but I think you are severely overestimating it.

• tailcalled 27 Mar 2015 19:16 UTC

  1 point

  in reply to: shminux’s comment on: Box­ing an AI?

  It will consider the possibility, but how will it find any evidence? As long as you keep the game lawful and simple, there would be no way to tell the difference between being in a game and being in reality.

• tailcalled 27 Mar 2015 19:28 UTC

  2 points

  in reply to: TheAncientGeek’s comment on: Values at com­pile time

  Maybe it would be clearer if we looked at some already existing maximization processes. Take for instance evolution. Evolution maximizes inclusive genetic fitness. You punish it by not donating sperm/​eggs. I don’t care, because evolution is not a personlike thing.

• tailcalled 27 Mar 2015 19:32 UTC

  1 point

  in reply to: Wes_W’s comment on: Box­ing an AI?

  Thinking isn’t magic. You need empiricism to find out if your thoughts are correct.

  Mere humans, exploring the space of possible hypotheses in a dramatically suboptimal way, have nevertheless hit upon the idea that we live in a simulation, and have ideas about how to confirm or disprove it.

  The ways people have invented to confirm or disprove that we live in a simulation are mostly bullshit and generally rely on the completely unrealistic assumption that the simulating universe looks a lot like our universe, in particular in terms of the way computation is done (on discrete processors, etc.)

• tailcalled 27 Mar 2015 19:43 UTC

  −2 points

  in reply to: Transfuturist’s comment on: Box­ing an AI?

  I would have assumed that we would let it learn about the real world, but I guess it’s correct that if enough information about the real world is hardcoded, my idea wouldn’t work.

  … which means my idea is an argument for minimizing how much is hardcoded into the AI, assuming the rest of the idea works.

• tailcalled 27 Mar 2015 20:47 UTC

  3 points

  in reply to: Wes_W’s comment on: Box­ing an AI?

  The idea isn’t to make a box that looks like our world, because, as you pointed out, that would be pretty unconvincing. The idea is to make a radically different and macroscopically slightly similar but much simpler world that it can be in.

  The purpose isn’t to make friendliness unnecessary but instead to test if the basics of the AI works even if we aren’t sure if it’s intelligent and possibly, depending on how the AI is designed, provide a space for testing friendliness. Just turning the AI on and seeing what happens would obviously be dangerous, hence boxing.

  If you’re just looking at ways to make the least-imperfect box you can, rather than claiming your box is perfect, I don’t think I’m actually disagreeing with you here.

  I’m claiming the box is perfect. You can’t escape from a prison if you don’t know it exists, and you can’t figure out it exists if it’s hidden in the laws of physics.

• tailcalled 27 Mar 2015 21:46 UTC

  2 points

  in reply to: Wes_W’s comment on: Box­ing an AI?

  Pick or design a game that contains some aspect of reality that you care about in terms of AI. All games have some element of learning, a lot have an element of planning and some even have varying degrees of programming.

  As an example, I will pick Factorio, a game that involves learning, planning and logistics. Wire up the AI to this game, with appropriate reward channels etc. etc.. Now you can test how good the AI is at getting stuff done; producing goods, killing aliens (which isn’t morally problematic, as the aliens don’t act as personlike morally relevant things) and generally learning about the universe.

  The step with morality depends on how the AI is designed. If it’s designed to use heuristics to identify a group of entities as humans and help them, you might get away with throwing it in a procedurally generated RPG. If it uses more general, actually morally relevant criteria (such as intelligence, self-awareness, etc.), you might need a very different setup.

  However, speculating at exactly what setup is needed for testing morality is probably very unproductive until we decide how we’re actually going to implement morality.