Liam Donovan

[Question] Is it good prac­tice to write ques­tions/​com­ments on old posts you’re try­ing to un­der­stand?

[Question] What’s been writ­ten about the na­ture of “son-of-CDT”?

• Liam Donovan 8 Dec 2019 17:28 UTC

  13 points

  in reply to: Larks’s comment on: There’s No Fire Alarm for Ar­tifi­cial Gen­eral Intelligence

  Apparently an LW user did a series of interviews with AI researchers in 2011, some of which included a similar question. I know most LW users have probably seen this, but I only found it today and thought it was worth flagging here.

[Question] What is the con­nec­tion be­tween these two defi­ni­tions of as­crip­tion uni­ver­sal­ity?

• Liam Donovan 27 Dec 2019 23:03 UTC

  12 points

  in reply to: Zack_M_Davis’s comment on: Funk-tunul’s Le­gacy; Or, The Le­gend of the Ex­tor­tion War

  Based on the quote from Jessica Taylor, it seems like the FDT agents are trying to maximize their long-term share of the population, rather than their absolute payoffs in a single generation? If I understand the model correctly, that means the FDT agents should try to maximize the ratio of FDT payoff : 9-bot payoff (to maximize the ratio of FDT:9-bot in the next generation). The algebra then shows that they should refuse to submit to 9-bots once the population of FDT agents gets high enough (Wolfram|Alpha link), without needing to drop the random encounters assumption.

  
  

  It still seems like CDT agents would behave the same way given the same goals, though?

• Liam Donovan 23 Oct 2019 15:39 UTC

  10 points

  in reply to: romeostevensit’s comment on: What are some un­pop­u­lar (non-nor­ma­tive) opinions that you hold?

  Euthanasia should be a universal right.

  This doesn’t sound non-normative at all?

• Liam Donovan 23 Oct 2019 16:12 UTC

  8 points

  in reply to: Zack_M_Davis’s comment on: What are some un­pop­u­lar (non-nor­ma­tive) opinions that you hold?

  I downvoted because I think the benefit of making stuff like this socially unacceptable on LW is higher than the cost of the OP getting one less response to their survey. The reasons it might be ” strong-downvote-worthy had it appeared in most other possible contexts” still apply here, and the costs of replacing it with a less-bad example seem fairly minimal.

• Liam Donovan 20 Jan 2020 20:38 UTC

  7 points

  in reply to: Zvi’s comment on: The Rocket Align­ment Problem

  For what it’s worth, I was just learning about the basics of MIRI’s research when this came out, and reading it made me less convinced of the value of MIRI’s research agenda. That’s not necessarily a major problem, since the expected change in belief after encountering a given post should be 0, and I already had a lot of trust in MIRI. However, I found this post by Jessica Taylor vastly clearer and more persuasive (it was written before “Rocket Alignment”, but I read “Rocket Alignment” first). In particular, I would expect AI researchers to be much more competent than the portrayal of spaceplane engineers in the post, and it wasn’t clear to me why the analogy should be strong Bayesian evidence for MIRI being correct.

• Liam Donovan 29 Dec 2019 7:09 UTC

  6 points

  in reply to: Liron’s comment on: Pro­gram­mers Should Plan For Lower Pay

  GDP is 2x higher than in 2000

  Why not use per capita real GDP (+25% since 2000)?

• Liam Donovan 16 Sep 2019 6:59 UTC

  6 points

  in reply to: Ofer’s comment on: A Cri­tique of Func­tional De­ci­sion Theory

  Yeah, wouldn’t someone following Guaranteed Payoffs as laid out in the post be unable to make credible promises?

• Liam Donovan 29 Aug 2019 5:05 UTC

  6 points

  on: [Link] Book Re­view: Refram­ing Su­per­in­tel­li­gence (SSC)

  
  

  “Ten years ago, everyone was talking about superintelligence, the singularity, the robot apocalypse. What happened?”

  What is this referencing? I was only 10 years old in 2009 but I have a strong impression that AI risk gets a lot more attention now than it did then.

  
  

  Also, what are the most salient differences between CAIS and the cluster of concepts Karnofsky and others were calling “Tool AI”?

[Question] Ev­i­dence other than evolu­tion for op­ti­miza­tion dae­mons?

• Liam Donovan 9 Dec 2019 4:14 UTC

  5 points

  in reply to: XiXiDu’s comment on: Q&A with Shane Legg on risks from AI

  Well, it’s been 8 years; how close are ML researchers to a “proto-AGI” with the capabilities listed? (embarassingly, I have no idea what the answer is)

• Liam Donovan 25 Oct 2019 17:04 UTC

  5 points

  in reply to: Kaj_Sotala’s comment on: bgae­sop’s Shortform

  From an epistemic rationality perspective, isn’t becoming less aware of your emotions and body a really bad thing? Not only does it give you false beliefs, but “not being in touch with your emotions/​body” is already a stereotyped pitfall for a rationalist to fall into…

• Liam Donovan 23 Oct 2019 17:53 UTC

  5 points

  in reply to: Zack_M_Davis’s comment on: What are some un­pop­u­lar (non-nor­ma­tive) opinions that you hold?

  (Keeping in mind that I have nothing to do with the inquiry and can’t speak for OP)

  Why is it desirable for the inquiry to turn up a representative sample of unpopular beliefs? If that were explicitly the goal, I would agree with you; I’d also agree (?) that questions with that goal shouldn’t be allowed. However, I thought the idea was to have some examples of unpopular opinions to use in a separate research study, rather than to directly research what unpopular beliefs LW holds.

  If the conclusion of the research turns out to be “here is a representative sample of unpopular LW beliefs: <a set of beliefs that doesn’t include anything too reactionary/​politically controversial>”, that would be a dishonest & unfortunate conclusion.

• Liam Donovan 27 Jun 2019 7:43 UTC

  5 points

  in reply to: David Scott Krueger (formerly: capybaralet)’s comment on: Open ques­tion: are min­i­mal cir­cuits dae­mon-free?

  Coincidentally I’m also trying to understand this post at the same time, and was somewhat confused by the “upstream”/​”downstream” distinction.

  What I eventually concluded was that there are 3 ways a daemon that intrinsically values optimizing some Y can “look like” it’s optimzing X:

  • Y = X (this seems both unconcerning and unlikely, and thus somewhat irrelevant)

  • optimzing Y causes optimization pressure to be applied to X (upstream daemon, describes humans if Y = our actual goals and X = inclusive genetic fitness)

  • The daemon is directly optimizing X because the daemon believes this instrumentally helps it achieve Y (downstream daemon, e.g. if optimizing X helps the daemon survive)

  
  

  Does this seem correct? In particular, I don’t understand why upstream daemons would have to have a relatively benign goal.

[Question] How does the or­ga­ni­za­tion “EthAGI” fit into the broader AI safety land­scape?

• Liam Donovan 9 Feb 2020 14:48 UTC

  4 points

  in reply to: noggin-scratcher’s comment on: [Link] Ig­no­rance, a skil­led practice

  Why are you calling this a nitpick? IMO it’s a major problem with the post—I was very unhappy that no mention was made of this obvious problem with the reasoning presented.

• Liam Donovan 2 Dec 2019 18:57 UTC

  4 points

  in reply to: Wei Dai’s comment on: The “Com­mit­ment Races” problem

  Would trying to become less confused about commitment races before building a superintelligent AI count as a metaphilosophical approach or a decision theoretic one (or neither)? I’m not sure I understand the dividing line between the two.

• Liam Donovan 19 Nov 2019 0:21 UTC

  4 points

  in reply to: habryka’s comment on: Matthew Walker’s “Why We Sleep” Is Rid­dled with Scien­tific and Fac­tual Errors

  Maybe the claim that climate scientists are liars? I don’t know if it’s true, but if I knew it were false I’d definitely downvote the post...