Liron

• Liron 8 Dec 2021 12:52 UTC

  2 points

  in reply to: adamShimi’s comment on: In­ter­pret­ing Yud­kowsky on Deep vs Shal­low Knowledge

  Ok I agree there are a bunch of important concepts to be aware of, such as complexity of value, and there are many ways for security mindset by itself to fail at flagging the extent of AI risk if one is ignorant of some of these other concepts.

  I just think the outside view and extrapolating trends is so far from how one should reason about mere nukes, and superhuman intelligence is very nuke-like or at least has a very high chance of being nuke-like: that is, unlock unprecedentedly large rapid irreversible effects. Extrapolating from current trends would have been quite unhelpful to nuclear safety. I know Eliezer is just trying to meet other people in the discussion where they are, but it would be nice to have another discussion that seems more on-topic from Eliezer’s own perspective.

• Liron 6 Dec 2021 16:09 UTC

  2 points

  in reply to: adamShimi’s comment on: In­ter­pret­ing Yud­kowsky on Deep vs Shal­low Knowledge

  For me the security mindset frame comes down to two questions:

  1. Can we always stop AI once we release it?

  2. Can we make the first unstoppable AI do what we want?

  To which I’d answer “no” and “only with lots of research”.

  Without security mindset, one tends to think an unstoppable AI is a-priori likely to do what humans want, since humans built it. With security mindset, one sees that most AIs are nukes that wreak havoc on human values, and getting them to do what humans want is analogous to building crash-proof software for a space probe, except the whole human race only gets to launch one probe and it goes to whoever launches it first.

  I’d like to see this kind of discussion with someone who doesn’t agree with MIRI’s sense of danger, in addition to all the discussions about how to extrapolate trends and predict development.

• Liron 6 Dec 2021 2:58 UTC

  10 points

  on: In­ter­pret­ing Yud­kowsky on Deep vs Shal­low Knowledge

  Besides invoking “Deep Knowledge” and the analogy of ruling out perpetual motion, another important tool for understanding AI foom risk is security mindset, which Eliezer has written about here.

  Maybe this is tangential, but I don’t get why the AI foom debate isn’t framed more often as a matter of basic security considerations. AI foom risk seems like a matter of basic security mindset. I think AI is a risk to humanity for the same reason I think any website can be taken out by a hack if you put a sufficiently large bounty on it.

  Humanity has all kinds of vulnerabilities that are exploitable by a team of fast simulated humans, not to mention Von Neumann simulations or superhuman AIs. There are so many plausible attack vectors by which to destroy or control humanity: psychology, financial markets, supply chains, biology, nanotechnology, just to name a few.

  It’s very plausible that the AI gets away from us, runs in the cloud, self-improves, and we can’t turn it off. It’s like a nuclear explosion that may start slow, but it’s picking up speed, recursively self-improving or even just speeding up to the level of an adversarial Von Neumann team, and it’s hidden in billions of devices.

  We have the example of nuclear weapons. The US was a singleton power for a few years due to developing nukes first. At least a nuclear explosion stops when it burns through its fissile material. AI doesn’t stop, and it’s a much more powerful adversary that will not be contained. It’s like the first nuclear pile you’re testing with has a yield much larger than Tsar Bomba. You try one test and then you’ve permanently crashed your ability to test.

  So to summarize my security-mindset view: Humanity is vulnerable to hackers, without much ability to restore a backup once we get hacked, and it’s very easy to think AI becomes a great hacker soon.

• Liron 1 Dec 2021 1:33 UTC

  16 points

  on: Soares, Tal­linn, and Yud­kowsky dis­cuss AGI cognition

  Jaan’s presentation illustrated important AI safety concepts more simply than I realized was possible. Great reference material!

  I find these dialogues interesting, informative and important, and hope they keep coming.

• Liron 1 Dec 2021 1:25 UTC

  4 points

  in reply to: Sammy Martin’s comment on: Soares, Tal­linn, and Yud­kowsky dis­cuss AGI cognition

  I think we gotta get the message out that consequentialism is a super-strong attractor.

• Liron 29 Nov 2021 12:32 UTC

  4 points

  in reply to: AllAmericanBreakfast’s comment on: Frame Control

  That might serve your purposes, and is at least better than simply giving into the speaker’s frame. But responding in a way that’s purely defensive, even if you do it in a disaffected way, means you can’t frame yourself as a polite socially-savvy party guest who’s in open communication with the whole group. The speaker may be intending to battle you away from that frame and monopolize it for themselves.

• Liron 29 Nov 2021 12:23 UTC

  5 points

  in reply to: blueiris’s comment on: Frame Control

  Geoff’s one tweet has enough context for me to see it as a likely example of frame control, so I think Aella has given adequate support for her claim, though it’s still possible to disagree with her analysis or ask for more supporting evidence.

• Liron 29 Nov 2021 4:53 UTC

  12 points

  in reply to: blueiris’s comment on: Frame Control

  You’re making strong personal claims about Aella and the commenters without providing enough evidence. By jumping to your conclusions and implying that the evidence is obvious, you’re violating community norms of politeness and process.

  Also, you clearly don’t know who you’re dealing with here. Many of us aren’t attracted to Aella’s gender. Many of us would speak up if we disagreed with Aella’s claims, even if we do like her and/​or are attracted to her. Many of us are naturally contrarian and disagreeable. Many of us want our comments to hold up well even if it’s revealed that the author of the post actually wasn’t Aella. Etc.

• Liron 28 Nov 2021 21:10 UTC

  2 points

  in reply to: Alexei’s comment on: Frame Control

  Oh nice, I think that’s good.

  I’d additionally frame this as a friendly interaction by politely (second-order condescendingly) adding, “You should continue though, I’ll try to get into it.” (Then you can keep using your phone without necessarily trying to listen.)

  The other person was attempting to put you in the frame of “impolite listener”. Your reply is a good way of blocking the attempt to frame you as consciously antagonistic. I would go further and frame myself as someone who is a polite listener and friend to everyone there, but also perfectly within my right to follow where my attention takes me.

  Regardless, I think the key to both of our approaches is that we can sound first-order chill so that nothing about our first-order words or tone of voice sounds antagonistic. It obeys the highest standard of polite behavior (ideally out-politing the other person), while the logical implication of our behavior is to assert our own frame against the frame control attempt.

• Liron 28 Nov 2021 20:55 UTC

  21 points

  on: The Ra­tion­al­ists of the 1950s (and be­fore) also called them­selves “Ra­tion­al­ists”

  Nice, thanks for sharing.

  I was just thinking how “rationalist” has been an increasingly positive-associated label that we can be proud to self-identify as, and it’s currently serving as a useful positive signal when others apply it to themselves (for me at least).

  The LW community and related rationalist clusters have been doing influential work in various fields: AI and AI safety, the theory and practice of science, journalism, effective altruism, entrepreneurship, investing, pedagogical fanfiction, and more. As a result, the term “rationalist” has been building momentum, and I expect we’ll increasingly see high-performers associating with it in various domains, the way “evidence-based” has gotten to the point of sounding good in every domain.

  I’ve also noticed that using the longer label “aspiring rationalist” is less common these days, and I’m glad we’re just settling on “rationalist”, since anyway rationalists are many things besides epistemically humble and goal-oriented.

• Liron 28 Nov 2021 12:45 UTC

  5 points

  in reply to: cmessinger’s comment on: Frame Control

  Lol interesting, that definitely undermines the person’s frame. The difference is that my version builds status/​respect for me as a capable frame battler while the other builds compassion for you as a victim of the person’s aggression (and a sufferer of the cat situation).

• Liron 28 Nov 2021 2:24 UTC

  14 points

  on: Frame Control

  Great post. I agree with your analysis. I especially like the part about how it often doesn’t help to try to judge the frame controller’s intent.

  FWIW, in the “Looks like I’m boring Aella” scenario, assuming I perceive the speaker to be overly aggressive in their frame control attempt, my move would be to politely disagree, except with a noticeable attitude of not buying into the frame that there’s anything wrong with me looking at my phone. I would reply with a tongue-in-cheek “No, this is all interesting stuff. Please continue.” where my voice is cheerful and encouraging but I’m still just looking at my phone.

• Liron 28 Nov 2021 0:47 UTC

  4 points

  on: [Linkpost] Be­ing Nor­mal by Brian Caplan

  Love this classification of weird-person manifestations:

  Manifestation #1: Saying Unconventional Things

  Manifestation #2: Doing Unconventional Actions

  Manifestation #3: The Integrity of Good

  Manifestation #4: The Integrity of Evil

• Liron 27 Nov 2021 21:47 UTC

  20 points

  in reply to: Matthew Barnett’s comment on: Yud­kowsky and Chris­ti­ano dis­cuss “Take­off Speeds”

  I think a superintelligent AI can FOOM its way to manufacturing nanobots because the biggest bottleneck to engineering and manufacturing those is research that can be done without needing input from the physical universe beyond the physics we already know, and the machines we already have, with very slight upgrades or creative usages beyond what they were designed for. Manufacturing nanobots is like a logic brain teaser for a sufficiently intelligent reasoner. I guess you have a different perspective in that you think the process requires a culture of socializing beings, and/​or more input from the physical universe?

• Liron 23 Nov 2021 11:59 UTC

  16 points

  in reply to: Daniel Kokotajlo’s comment on: Yud­kowsky and Chris­ti­ano dis­cuss “Take­off Speeds”

  While this may not be the ideal format for it, I thought Eliezer’s voicing of despair was a useful update to publish to the LW community about the current state of his AI beliefs.

• Liron 21 Nov 2021 18:29 UTC

  2 points

  in reply to: jimrandomh’s comment on: Im­prov­ing on the Karma System

  Whoa I like the labeled upvotes and downvotes. It’s like the emoji reactions feature but for rational discourse.

• Liron 21 Nov 2021 14:05 UTC

  3 points

  on: Dat­ing pro­files from first prin­ci­ples: het­ero­sex­ual male pro­file design

  This is a great post. I fully endorse its advice.

  FWIW, I’ve spent a lot of time thinking about how to optimize online dating, and run a business called RelationshipHero.com which offers coaching on this topic (among other topics)

Naval Ravikant and Chris Dixon Didn’t Ex­plain Any Web3 Use Cases

• Liron 2 Oct 2021 11:14 UTC

  9 points

  on: The LessWrong Team is now Light­cone In­fras­truc­ture, come work with us!

  Amazing name

• Liron 29 Sep 2021 13:08 UTC

  22 points

  on: A re­view of Steven Pinker’s new book on rationality

  I listened the audiobook and fully endorse this review. It’s much better than what I would have written.

  I really love Pinker’s other books so I was looking forward to this but unfortunately I already had all the fun spoiled by reading LW sequences, like literally all. The Sequences are a superset of this book, longer and quirkier, deeper and more insightful. But I agree that Pinker’s book is a good fit for someone who wants a more compact and mainstream-sounding intro to rationality.