PeterMcCluskey

• PeterMcCluskey 29 Nov 2022 4:02 UTC

  3 points

  0 ∶ 0

  in reply to: jacob_cannell’s comment on: Re­view: LOVE in a simbox

  I don’t see why alignment inherently needs more realism than intelligence

  I was focused solely on testing alignment. I’m pretty confused about how much realism is needed to produce alignment.

  explicitly we are designing sims with dualistic physics (mind is a fully separate non matter-based phenomenon)

  I guess I should have realized that, but it did not seem obvious enough for me to notice.

  So it doesn’t seem that much any extra effort is required to get reasonable sized populations of agents

  I’m skeptical. I don’t know whether I’ll manage to quantify my intuitions well enough to figure out how much we disagree here.

  The AGI in simboxes shouldn’t be aware of concepts like utility functions, let alone the developers or the developers modifying their utility functions (or minds).

  It seems likely that some AGIs would notice changes in behavior. I expect it to be hard to predict what they’ll infer.

  But now that I’ve thought a bit more about this, I don’t see any likely path to an AGI finding a way to resist the change in utility function.

Re­view: LOVE in a simbox

• PeterMcCluskey 26 Nov 2022 4:06 UTC

  12 points

  1 ∶ 0

  on: Semi-con­duc­tor/​AI Stock Dis­cus­sion.

  I’ve got modest positions in GOOGL, MSFT.

  I’ve got a bit more than 5% of my portfolio in semiconductor and related stocks: KLAC, LSE:SMSN, MTRN, AOSL, ASYS, AMKR, TRT, SCIA. I’m likely to buy more sometime in the next year, but I’m being patient because we’re likely at a poor part of an industry cycle.

  Robotics seem likely to benefit from AI. I suspect the main winners will be companies that aren’t yet public, as I’m not too impressed by the opportunities I see so far. I’m playing this mainly via tiny positions in LIDAR companies (INVZ, OUST, LAZR) and SYM.

  I have a modest position in OSS.

  I tried to invest in Conjecture, but they don’t seem interested in small investors.

• PeterMcCluskey 14 Nov 2022 16:30 UTC

  2 points

  0 ∶ 0

  in reply to: Linda Linsefors’s comment on: Why don’t we have self driv­ing cars yet?

  I suspect that testing is one of the more important bottlenecks.

  I suspect that some current systems are safe enough if their caution is dialed up to where they’re annoyingly slow 2% of the time, and that leaves them not quite reliable enough at reaching a destination to be competitive.

• PeterMcCluskey 9 Nov 2022 0:18 UTC

  4 points

  0 ∶ 0

  in reply to: Eliezer Yudkowsky’s comment on: Ap­ply­ing su­per­in­tel­li­gence with­out col­lu­sion

  so long as their utility functions aren’t negatives of each other (or equally exotic in some other way)

  Why doesn’t setting some of the utility functions to red-team the others make them sufficiently antagonistic?

• PeterMcCluskey 20 Oct 2022 18:29 UTC

  13 points

  3 ∶ 0

  on: The her­i­ta­bil­ity of hu­man val­ues: A be­hav­ior ge­netic cri­tique of Shard Theory

  I agree with you at least 90% about what is heritable.

  Yet I don’t see how this says that anything is wrong with shard theory. It seems fairly plausible that the cortex is randomly initialized, and the effects of genetic differences on the cortex are indirect, via things such as subcortical hardwiring.

  Or even something less direct. Playing basketball seems rather heritable. That might be mostly due to the influence of genes on height.

  I see very little in your post that says anything about what is hard-coded.

• PeterMcCluskey 12 Oct 2022 16:03 UTC

  7 points

  0 ∶ 0

  in reply to: MathiasKB’s comment on: Ac­tu­ally, All Nu­clear Famine Papers are Bunk

  ALLFED has been doing research recently into nuclear winter. This seems to be their most relevant publication so far. I haven’t read it yet.

• PeterMcCluskey 8 Oct 2022 20:00 UTC

  4 points

  1 ∶ 1

  on: So, geez there’s a lot of AI con­tent these days

  I’m trying to shift my focus more toward AI, due to the likelihood that it will have big impacts over the next decade.

  I’d like newbies to see some encouragement to attend a CFAR workshop. But there’s not much new to say on that topic, so it’s hard to direct people’s attention there.

• PeterMcCluskey 13 Sep 2022 16:16 UTC

  2 points

  on: Briefly think­ing through some analogs of debate

  The default outcome of debate does not look promising. But there’s a good deal of room to improve on the default.

  Maybe half the problem with public discourse is that people have social goals that distract them from reality. I’m not confident that AI researchers will be more truth-oriented, but I see plenty of room for hope.

  Drexler’s CAIS paper describes some approaches that are likely needed to make debate work: Section 25:

  Optimized advice need not be optimized to induce its acceptance Advice optimized to produce results may be manipulative, optimized to induce a client’s acceptance; advice optimized to produce results conditioned on its acceptance will be neutral in this regard.

  Section 20:

  Collusion among superintelligent oracles can readily be avoided

  C1) To improve the quality of answers, it is natural to implement multiple, diverse (and implicitly competing) systems to propose alternatives.
  C2) To identify low-quality or misleading answers, it is natural to employ diverse critics, any one of which could disrupt deceptive collusion.
  C3) Systems of diverse, competing proposers and critics naturally implement both independent and adversarial objectives.
  C4) It is natural to apply fixed (hence memory-free) system instantiations to multiple problems, incidentally yielding a series of history-blind, single-move decisions.
  C5) It is natural to provide differentiated, task-relevant information to systems solving different problems, typically omitting knowledge of general circumstances.

  Some of these approaches are costly to implement. That might doom debate.

  Success with debate likely depends on the complexity of key issues to be settled by debate, and/​or the difficulty of empirically checking proposals.

  Eliezer sometimes talks as if we’d be stuck evaluating proposals that are way too complex for humans to fully understand. I expect alignment can be achieved by evaluating some relatively simple, high-level principles. I expect we can reject proposals from AI debaters that are too complex, and select simpler proposals until we can understand them fairly well. But I won’t be surprised if we’re still plagued by doubts at the key junctures.

• PeterMcCluskey 12 Sep 2022 16:28 UTC

  2 points

  1 ∶ 1

  on: Ar­gu­ment against 20% GDP growth from AI within 10 years [Linkpost]

  If Fen is using the most appropriate basic approach to forecasting growth, then his conclusions are correct.

  Fen does not seem to be addressing the kind of models used in this OpenPhil report.

• PeterMcCluskey 4 Aug 2022 16:47 UTC

  4 points

  0 ∶ 0

  in reply to: Adam Scholl’s comment on: Two-year up­date on my per­sonal AI timelines

  I see a difference between finding a new core of intelligence versus gradual improvements to the core(s) of intelligence that we’re already familiar with.

• PeterMcCluskey 1 Aug 2022 21:07 UTC

  2 points

  0 ∶ 0

  in reply to: Charlie Steiner’s comment on: Drexler’s Nan­otech Forecast

  Current tools do not seem adequate for directly building Drexler’s designs. That doesn’t mean Drexler’s designs are impossible.

  The first few generations of molecular assemblers will likely use diffusion and small energy differences. They’ll be hard to use, and will break frequently. They’ll be used to build tools to roughly atomic precision. Those tools will bootstrap numerous generations of systems which get closer to Drexler’s designs. With enough trial and error, we’ll eventually find ways to handle the problems you describe.

  I won’t bet much money that it’s easier than the Manhattan Project. I’m moderately confident that nanotech needs less total resources, but it likely needs a larger number of smart people doing lots of exploration.

• PeterMcCluskey 1 Aug 2022 21:06 UTC

  2 points

  0 ∶ 0

  in reply to: Gerald Monroe’s comment on: Drexler’s Nan­otech Forecast

  The “mechanism” you describe for AGI does not at all sound like something that will produce results within any predictable time.

Drexler’s Nan­otech Forecast

• PeterMcCluskey 28 Jul 2022 19:23 UTC

  11 points

  3 ∶ 0

  on: Covid 7/​28/​22: Ruin­ing It For Everyone

  What should ‘Theranos level fraud’ mean? That some influential people manage to focus a lot of press attention on a few culprits?

  The problems with the amyloid hypothesis have been visible enough for at least a decade that the blame should be spread wider than just one bad paper.

  The total harm from amyloid groupthink is much bigger than Theranos, but the harm from the paper might well be merely Theranos level.

• PeterMcCluskey 25 Jul 2022 16:18 UTC

  2 points

  in reply to: Alex_Altair’s comment on: MIRI an­nounces new “Death With Dig­nity” strategy

  My initial reaction to the post was almost as negative as yours.

  I’ve partly changed my mind, due to this steelman of Eliezer’s key point by Connor Leahy.

• PeterMcCluskey 23 Jul 2022 3:09 UTC

  9 points

  0 ∶ 3

  on: Chang­ing the world through slack & hobbies

  This seems to nudge people in a generally good direction.

  But the emphasis on slack seems somewhat overdone.

  My impression is that people who accomplish the most typically have had small to moderate amounts of slack. They made good use of their time by prioritizing their exploration of neglected questions well. That might create the impression of much slack, but I don’t see slack as a good description of the cause.

  One of my earliest memories of Eliezer is him writing something to the effect that he didn’t have time to be a teenager (probably on the Extropians list, but I haven’t found it).

  I don’t like the way you classify your approach as an alternative to direct work. I prefer to think of it as a typical way to get into direct work.

  I’ve heard a couple of people mention recently that AI safety is constrained by the shortage of mentors for PhD theses. That seems wrong. I hope people don’t treat a PhD as a standard path to direct work.

  I also endorse Anna’s related comments here.

• PeterMcCluskey 23 Jul 2022 0:39 UTC

  3 points

  0 ∶ 0

  in reply to: Ajeya Cotra’s comment on: Without spe­cific coun­ter­mea­sures, the eas­iest path to trans­for­ma­tive AI likely leads to AI takeover

  Okay, it sounds like you know more about that than I do. It sounds like it might cause Alex to typically care about outcomes that are months in the future? That seems somewhat close to what it would take to be a major danger.

• PeterMcCluskey 20 Jul 2022 0:19 UTC

  12 points

  1 ∶ 2

  on: Without spe­cific coun­ter­mea­sures, the eas­iest path to trans­for­ma­tive AI likely leads to AI takeover

  How far-sighted would Alex be?

  My guess is that the default outcome is that it’s easier to train Alex to care about near-term results, and harder to get Alex to care about the distant future. Especially in a “racing forward” scenario, I’d expect Magma to prioritize training that can be done quickly.

  So I expect the most likely result is for Alex to be more myopic than the humans who run Magma. This seems somewhat at odds with capabilities you imply.

  That guess is heavily dependent on details that you haven’t specified. There are likely ways that Alex could be made more far-sighted than humans. CarlShulman’s point about retroactive punishment suggests one possibility, but it doesn’t seem close to being the default.

  I’m not suggesting you add those details. The post is a bit long as it is. My advice is to be more uncertain than you are (at least in the introduction to this post) whether the default outcome is AI takeover. That’s a minor complaint about a mostly great post.

• PeterMcCluskey 18 Jul 2022 2:53 UTC

  6 points

  0 ∶ 0

  in reply to: Algon’s comment on: QNR Prospects

  You’re partly on the right track.

  If QNRs produced by disparate format are used to train a neural net, then I’d guess they’d be translated to a common format first.

  There would likely be multiple models of books, some generated with human guidance, and others generated to optimize a variety of predictions.

  Maybe some models would be optimized for answering: what does the author believe about X?, as evaluated by a service that’s designed to evaluate those answers.

  Some models might be constructed mostly by a system that takes info about the reputations of works that the book cites, and infers reliability estimates for each of the book’s claims, by aggregating the reliability of the citations that support each claim.

  Possibly you’re confused because you’re imagining a more restrictive set of rules than Drexler intends for composing QNRs. He’s using rules that are much more general than what’s typically used for creating syntax trees of natural language. See section 8.1.2 for some hints. But I don’t see a clear answer to this kind of confusion.