UmamiSalami

Notes on the Safety in Ar­tifi­cial In­tel­li­gence conference

• UmamiSalami 22 Apr 2017 4:25 UTC

  14 points

  in reply to: Fluttershy’s comment on: Effec­tive al­tru­ism is self-recommending

  It seems to me that Givewell has already acknowledged perfectly well that VillageReach is not a top effective charity. It also seems to me that there’s lots of reasons one might take GiveWell’s recommendations seriously, and that getting “particularly horrified” about their decision not to research exactly how much impact their wrong choice didn’t have is a rather poor way to conduct any sort of inquiry on the accuracy of organizations’ decisions.

• UmamiSalami 3 Jul 2016 8:08 UTC

  6 points

  on: Zom­bies Redacted

  This was longer than it needed to be, and in my opinion, somewhat mistaken.

  The zombie argument is not an argument for epiphenomenalism, it’s an argument against physicalism. It doesn’t assume that interactionist dualism is false, regardless of the fact that Chalmers happens to be an epiphenomenalist.

  Chalmers furthermore specifies that this true stuff of consciousness is epiphenomenal, without causal potency—but why say that?

  Maybe because interactionism violates the laws of physics and is somewhat at odds with everything we (think we) know about cognition. There may be other arguments as well. It has mostly fallen out of favor. I don’t know the specific reasons why Chalmers rejects it.

  Once you see the collision between the general rule that consciousness has no effect, to the specific implication that consciousness has no effect on how you think about consciousness (in any way that affects your internal narrative that you could choose to say out loud), zombie-ism stops being intuitive. It starts requiring you to postulate strange things.

  In the epiphenomenalist view, for whatever evolutionary reason, we developed to have discussions and beliefs in rich inner lives. Maybe those thoughts and discussions help us with being altruistic, or maybe they’re a necessary part of our own activity. Maybe the illusion of interactionism is necessary for us to have complex cognition and decisionmaking.

  Also in the epiphenomenalist view, psychophysical laws relate mental states to neurophysical aspects of our cognition. So for some reason there is a relation between acting/​thinking of pain, and mental states which are painful. It’s not arbitrary or coincidental because the mental reaction to pain (dislike/​avoid) is a mirror of the physical reaction to pain (express dislike/​do things to avoid it).

  But Chalmers just wrote all that stuff down, in his very physical book, and so did the zombie-Chalmers.

  Chalmers isn’t denying that the zombie Chalmers would write that stuff down. He’s denying that its beliefs would be justified. Maybe there’s a version of me in a parallel universe that doesn’t know anything about philosophy but is forced to type certain combinations of letters at gunpoint—that doesn’t mean that I don’t have reasons to believe the same things about philosophy in this universe.

• UmamiSalami 1 Nov 2016 17:43 UTC

  5 points

  in reply to: [deleted]’s comment on: Is Global Re­in­force­ment Learn­ing (RL) a Fan­tasy?

  I came here to write exactly what gjm said, and your response is only to repeat the assertion “Scenarios in which the AI Danger comes from an AGI that is assumed to be an RL system are so ubiquitous that it is almost impossible to find a scenario that does not, when push comes to shove, make that assumption.”

  What? What about all the scenarios in IEM or Superintelligence? Omohundro’s paper on instrumental drives? I can’t think of anything which even mentions RL, and I can’t see how any of it relies upon such an assumption.

  So you’re alleging that deep down people are implicitly assuming RL even though they don’t say it, but I don’t see why they would need to do this for their claims to work nor have I seen any examples of it.

• UmamiSalami 24 Apr 2014 2:00 UTC

  5 points

  on: Open thread, 21-27 April 2014

  Sorry if this has topic has been beaten to death already here. I was wondering if anyone here has seen this paper and has an opinion on it.

  The abstract: “This paper argues that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a “posthuman” stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed.”

  Quite simple, really, but I found it extremely interesting.

  http://​​people.uncw.edu/​​guinnc/​​courses/​​Spring11/​​517/​​Simulation.pdf

• UmamiSalami 2 Nov 2016 1:08 UTC

  4 points

  in reply to: [deleted]’s comment on: Is Global Re­in­force­ment Learn­ing (RL) a Fan­tasy?

  All of the architectures assumed by people who promote these scenarios have a core set of fundamental weaknesses (spelled out in my 2014 AAAI Spring Symposium paper).

  The idea of superintelligence at stake isn’t “good at inferring what people want and then decides to do what people want,” it’s “competent at changing the environment”. And if you program an explicit definition of ‘happiness’ into a machine, its definition of what it wants—human happiness—is not going to change no matter how competent it becomes. And there is no reason to expect that increases in competency lead to changes in values. Sure, it might be pretty easy to teach it the difference between actual human happiness and smiley faces, but it’s a simplified example to demonstrate a broader point. You can rephrase it as “fulfill the intentions of programmers”, but then you just kick things back a level with what you mean by “intentions”, another concept which can be hacked, and so on.

  Your argument for “swarm relaxation intelligence” is strange, as there is only one example of intelligence evolving to approximate the format you describe (not seven billion—human brains are conditionally dependent, obviously), and it’s not even clear that human intelligence isn’t equally well described as goal directed agency which optimizes for a premodern environment. The arguments in Basic AI Drives and other places don’t say anything about how AI will be engineered, so they don’t say anything about whether they’re driven by logic, just about how it will behave, and all sorts of agents behave in generally logical ways without having explicit functions to do so. You can optimize without having any particular arrangement of machinery (humans do as well).

  Anyway, in the future when making claims like this, it would be helpful to make it clear early on that you’re not really responding to the arguments that AI safety research relies upon—you’re responding to an alleged set of responses to the particular responses that you have given to AI safety research.

  That is why I said what I said. We discussed it at the 2014 Symposium If I recall correctly Steve used that strategy (although to be fair I do not know how long he stuck it out). I know for sure that Daniel Dewey used the Resort-to-RL maneuver, because that was the last thing he was saying as I had to leave the meeting.

  So you had two conversations. I suppose I’m just not convinced that there is an issue here: I think most people would probably reject the claims in your paper in the first place, rather than accepting them and trying a different route.

• UmamiSalami 27 Aug 2016 4:11 UTC

  3 points

  on: Hedging

  It seems like hedging is the sort of thing which tends to make the writer sound more educated and intelligent, if possibly more pretentious.

• UmamiSalami 1 Aug 2017 1:41 UTC

  2 points

  on: What Are The Chances of Ac­tu­ally Achiev­ing FAI?

  I think it’s about a 0.75 probability, conditional upon smarter-than-human AI being developed. Guess I’m kind of an optimist. TL;DR I don’t think it will be very difficult to impart your intentions into a sufficiently advanced machine.

• UmamiSalami 22 Apr 2017 5:08 UTC

  2 points

  in reply to: Benquo’s comment on: Effec­tive al­tru­ism is self-recommending

  Why do you expect that to be true?

  Because they generally emphasize these values and practices when others don’t, and because they are part of a common tribe.

  How strongly? (“Ceteris paribus” could be consistent with an extremely weak effect.) Under what criterion for classifying people as EAs or non-EAs?

  Somewhat weakly, but not extremely weakly. Obviously there is no single clear criteria, it’s just about people’s philosophical values and individual commitment. At most, I think that being a solid EA is about as important as having a couple additional years of relevant experience or schooling.

  I do think that if you had a research-focused organization where everyone was an EA, it would be better to hire outsiders at the margin, because of the problems associated with homogeneity. (This wouldn’t the case for community-focused organizations.) I guess it just depends on where they are right now, which I’m not too sure about. If you’re only going to have 1 person doing the work, e.g. with an EA fund, then it’s better for it to be done by an EA.

• UmamiSalami 22 Apr 2017 4:53 UTC

  2 points

  in reply to: Benquo’s comment on: Effec­tive al­tru­ism is self-recommending

  I bet that most of the people who donated to Givewell’s top charities were, for all intents and purposes, assuming their effectiveness in the first place. From the donor end, there were assumptions being made either way (and there must be; it’s impractical to do all kinds of evaluation on one’s own).

• UmamiSalami 22 Apr 2017 4:40 UTC

  2 points

  in reply to: Benquo’s comment on: Effec­tive al­tru­ism is self-recommending

  I think EA is something very distinct in itself. I do think that, ceteris paribus, it would be better to have a fund run by an EA than a fund not run by an EA. Firstly, I have a greater expectation for EAs to trust each other, engage in moral trades, be rational and charitable about each other’s points of view, and maintain civil and constructive dialogue than I do for other people. And secondly, EA simply has the right values. It’s a good culture to spread, which involves more individual responsibility and more philosophical clarity. Right now it’s embryonic enough that everything is tied closely together. I tentatively agree that that is not desirable. But ideally, growth of thoroughly EA institutions should lead to specialization and independence. This will lead to a much more interesting ecosystem than if the intellectual work is largely outsourced.

• UmamiSalami 11 Oct 2016 18:33 UTC

  2 points

  in reply to: cunning_moralist’s comment on: Rea­son­able Re­quire­ments of any Mo­ral Theory

  Among hedonistic utilitarians it’s quite normal to demand both completeness

  Utilitarianism provides no guidance on many decisions: any decision where both actions produce the same utility.

  Even if it is a complete theory, I don’t think that completeness is demanded of the theory; rather it’s merely a tenet of it. I can’t think of any good a priori reasons to expect a theory to be complete in the first place.

• UmamiSalami 1 Jul 2016 16:38 UTC

  2 points

  in reply to: The_Jaded_One’s comment on: Notes on the Safety in Ar­tifi­cial In­tel­li­gence conference

  In fairness, I didn’t directly ask any of them about it, and it wasn’t really discussed. There could have been some who had read the relevant work, and many who believed it to be reasonable, but just didn’t happen to speak up during the presentations or in any of the conversations I was in.

• UmamiSalami 9 Jan 2017 23:58 UTC

  1 point

  on: If I must eat meat, I eat pork

  pigs strike a balance between the lower suffering, higher ecological impact of beef and the higher suffering, lower ecological impact of chicken.

  This was my thinking for coming to the same conclusion. But I am not confident in it. Just because something minimaxes between two criteria doesn’t mean that it minimizes overall expected harm.

• UmamiSalami 27 Sep 2016 5:23 UTC

  1 point

  in reply to: Ozyrus’s comment on: Open thread, Sep. 26 - Oct. 02, 2016

  See Omohundro’s paper on convergent instrumental drives

• UmamiSalami 27 Jul 2016 15:28 UTC

  1 point

  in reply to: naasking’s comment on: Zom­bies Redacted

  It’s unjustified in the same way that vilalism was an unjustified explanation of life: it’s purely a product of our ignorance.

  It’s not. Suppose that the ignorance went away: a complete physical explanation of each of our qualia—“the redness of red comes from these neurons in this part of the brain, the sound of birds flapping their wings is determined by the structure of electric signals in this region,” and so on—would do nothing to remove our intuitions about consciousness. But a complete mechanistic explanation of how organ systems work would (and did) remove the intuitions behind vitalism.

  I disagree. You’ve said that epiphenominalists hold that having first-hand knowledge is not causally related to our conception and discussion of first-hand knowledge. This premise has no firm justification.

  Well… that’s just what is implied by epiphenomenalism, so the justification for it is whatever reasons we have to believe epiphenomenalism in the first place. (Though most people who gravitate towards epiphenomenalism seem to do so out of the conviction that none of the alternatives work.)

  Denying it yields my original argument of inconceivability via the p-zombie world.

  As I’ve said already, your argument can’t show that zombies are inconceivable. It only attempts to show that an epiphenomenalist world is probabilistically implausible. These are very different things.

  Accepting it requires multiplying entities unnecessarily, for if such knowledge is not causally efficacious

  Well the purpose of rational inquiry is to determine which theories are true, not which theories have the fewest entities. Anyone who rejects solipsism is multiplying entities unnecessarily.

  I previously asked for any example of knowledge that was not a permutation of properties previously observed.

  I don’t see why this should matter for the zombie argument or for epiphenomenalism. In the post where you originally asked this, you were confused about the contextual usage and meaning behind the term ‘knowledge.’

• UmamiSalami 29 Nov 2015 22:59 UTC

  1 point

  in reply to: diegocaleiro’s comment on: The Growth of My Pes­simism: Tran­shu­man­ism, Im­mor­tal­ism, Effec­tive Altru­ism.

  So for us to understand what you’re even trying to say, you want us to read a bunch of articles, talk to one of your friends, listen to a speech, and only then will we become EAs good enough for you? No thanks.

• UmamiSalami 1 Aug 2017 1:42 UTC

  0 points

  in reply to: Furcas’s comment on: What Are The Chances of Ac­tu­ally Achiev­ing FAI?

  1%? Shouldn’t your basic uncertainty over models and paradigms be great enough to increase that substantially?

• UmamiSalami 28 Apr 2017 13:31 UTC

  0 points

  in reply to: Viliam’s comment on: Effec­tive al­tru­ism is self-recommending

  I haven’t seen any parts of Givewell’s analyses that involve looking for the right buzzwords. Of course, it’s possible that certain buzzwords subconsciously manipulate people at Givewell in certain ways, but the same can be said for any group, because every group has some sort of values.

• UmamiSalami 19 Mar 2017 4:30 UTC

  0 points

  on: Could util­ity func­tions be for nar­row AI only, and down­right an­ti­thet­i­cal to AGI?

  In fact, it seems to me that the less intelligent an organism is, the easier its behavior can be approximated with model that has a utility function!

  Only because those organisms have fewer behaviors in general. If you put a human in an environment where its options and sensory inputs were as simple as those experienced by apes and cats, humans would probably look like equally simple utility maximizers.