Trevor1

[Question] Did I just catch GPTchat do­ing some­thing un­ex­pect­edly in­sight­ful?

• Trevor1 2 Dec 2022 0:23 UTC

  4 points

  1 ∶ 0

  in reply to: the gears to ascenscion’s comment on: Wired Magaz­ine prints mis­in­for­ma­tion about AI safety

  Mischaracterizations, misleading language, and false dichotomies count as misinformation. Just because it’s prevalent on the modern internet doesn’t change the fact that it misdirects people in a tangential direction away from having accurate models of reality.

  What makes internet content misinformation is about how manipulative and misleading their piece is, not about plausible deniability that the author could have unintentionally gotten something wrong or thinking suboptimal thoughts. Real life interactions have lower standards for misinformation, because the internet contains massive billion-dollar industries for lying to people at large scale, with the industry systematically being optimized to make the authors and outlets immune to accusations of outright lying.

• Trevor1 2 Dec 2022 0:02 UTC

  14 points

  2 ∶ 1

  on: A challenge for AGI or­ga­ni­za­tions, and a challenge for readers

  Eliezer also hereby gives a challenge to the reader: Eliezer and Nate are thinking about writing up their thoughts at some point about OpenAI’s plan of using AI to aid AI alignment. We want you to write up your own unanchored thoughts on the OpenAI plan first, focusing on the most important and decision-relevant factors, with the intent of rendering our posting on this topic superfluous.

  Our hope is that challenges like this will test how superfluous we are, and also move the world toward a state where we’re more superfluous /​ there’s more redundancy in the field when it comes to generating ideas and critiques that would be lethal for the world to never notice.

  I strongly endorse this, based on previous personal experience with this sort of thing. Crowdsourcing routinely fails at many things, but this isn’t one of them (it does not routinely fail).

  It’s a huge relief to see that there are finally some winning strategies, lately there’s been a huge scarcity of those.

Wired Magaz­ine prints mis­in­for­ma­tion about AI safety

• Trevor1 26 Nov 2022 21:02 UTC

  8 points

  1 ∶ 2

  on: What videos should Ra­tional An­i­ma­tions make?

• Trevor1 26 Nov 2022 6:45 UTC

  6 points

  2 ∶ 0

  on: Semi-con­duc­tor/​AI Stock Dis­cus­sion.

  I don’t use twitter because social media use seems like a losing strategy for living in the world over the next 10 years. Is there any alternative such as an email newsletter?

• Trevor1 23 Nov 2022 19:17 UTC

  6 points

  0 ∶ 0

  on: An­nounc­ing AI safety Men­tors and Mentees

  Does AI policy count, or is this more technical AI safety? I do a specific niche in AI policy so my knowledge is somewhat lopsided, and I have the credentials to prepare people for AI policy but I’d definitely like to meet people and get industry advice.

[Question] What is the best source to ex­plain short AI timelines to a skep­ti­cal per­son?

• Trevor1 22 Nov 2022 3:14 UTC

  5 points

  0 ∶ 0

  in reply to: SurfingOrca’s comment on: What is the best ar­ti­cle to in­tro­duce some­one to AI safety for the first time?

  Yes, that was me too! But it’s a bit long.

[Question] What is the best ar­ti­cle to in­tro­duce some­one to AI safety for the first time?

[Question] Is Ger­man ra­tio­nal­ity more memet­i­cally de­rived from the west or former east?

• Trevor1 21 Nov 2022 5:30 UTC

  4 points

  0 ∶ 0

  in reply to: swarriner’s comment on: Is Crypto built on a foun­da­tion of ig­no­rance about In­for­ma­tion Asym­me­try?

  This is a really good way of putting it, and I’ve never thought about it this effectively before. Before reading this, my best thoughts like this on crypto were comparing cryptocoins to gold reserves or corn or oil, which are difficult to flood the market with. But, like, you might end up disappointed if a new cold war starts and world gold reserves get sold off. But that risk is nothing compared to crypto exchanges tanking the industry or everything related to hacking. If AI becomes a fire hose of cognitive labor that makes everything topsy turvy then that’s even worse for crypto.

[Question] Is Crypto built on a foun­da­tion of ig­no­rance about In­for­ma­tion Asym­me­try?

• Trevor1 16 Nov 2022 21:51 UTC

  4 points

  0 ∶ 1

  on: Is there some rea­son LLMs haven’t seen broader use?

  I don’t know about big reasoning machines, but I’ve heard a lot of rumors about LLMs being integrated into an extremely wide variety of extant ML systems that were already commercially viable on their own. It seems pretty intuitive to me that LLMs can provide some very good layers to support other systems. What have people heard about that?

• Trevor1 16 Nov 2022 20:50 UTC

  4 points

  0 ∶ 1

  in reply to: Ben Pace’s comment on: Care­fully-ed­ited Tran­script of Trump’s Nov 15th Cam­paign Announcement

  That’s actually the reason why I put the disclaimer on, since it’s definitely annoying for some people but also extremely helpful for people who do AI policy, foreign affairs, and nuclear risk analysis. I understand that the Trump title might repel some people from the forum, but the post is pretty unambiguously for professional purposes and not for political discussion. I don’t expect it to be on the front page for more than 12 hours anyway.

  Ideally I’d put it on EAforum since this is mainly a rationalist forum not an AI policy forum, but there’s some serious problems with that for a variety of reasons. I can trust the forecasting/​policy community on Lesswrong the way I can’t trust the people on EAforum. Also, it might be AI policy but I’m definitely not putting it on alignment forum, for exactly the reasons you’ve stated above.

Care­fully-ed­ited Tran­script of Trump’s Nov 15th Cam­paign Announcement

• Trevor1 15 Nov 2022 5:44 UTC

  9 points

  0 ∶ 0

  on: We need bet­ter pre­dic­tion markets

  A prediction market is only as good as the participants. People with the insider information and highly lucrative information necessary to forecast FTX-like catastrophes generally don’t participate in prediction markets, especially not free ones, and probably won’t any time soon. That’s something that financial and analyst firms can do, because they have ways to guarantee that their org’s members are safe, and that they won’t get targeted by all sorts of vicious people out there.

  Startups are fundamentally incapable of this. Wall street is highly capable of this, possibly also legal and government contractors that I’m not aware of, but it’s only possible because those systems are fundamentally closed and ingrained inside large nontransparent institutions that have built up networks of trust for decades.

• Trevor1 13 Nov 2022 20:44 UTC

  9 points

  2 ∶ 0

  in reply to: TekhneMakre’s comment on: The Align­ment Com­mu­nity Is Cul­turally Broken

  I just want to clarify that there are also “create more alignment researchers” people, not just “buy time” people and “technical alignment” people. I am legally and morally obligated to avoid anything related to “buying time”. And I also don’t touch it with a ten foot pole because it seems much much much easier and safer to double the number of people working on alignment than to halve the annual R&D of the global AI industry.

• Trevor1 13 Nov 2022 13:40 UTC

  6 points

  3 ∶ 1

  on: Not­ing an un­sub­stan­ti­ated com­mu­nal be­lief about the FTX disaster

  It seems like we’re all getting distracted from the main point here. It doesn’t even matter whether SBF did it, let alone why. What matters is what this says about the kind of world we live in, for the last 20 years, and, now, for the last 7 days:

  I strongly suspect^[4] that in ten years from now, conventional wisdom will hold the above belief as being basically cannon, regardless of further evidence in either direction. This is because it presents an intrinsically interesting, almost Hollywood villain-esque narrative, one that will surely evoke endless “hot takes” which journalists, bloggers, etc. will have a hard time passing over. Expect this to become the default understanding of what happened (from outsiders at least), and prepare accordingly.

  The fact that Lesswrong is vulnerable to this, let alone EA, is deeply disturbing. Smart people are supposed to automatically coordinate around this sort of thing, because that’s what agents do, and that’s not what’s happening right now. This is basically a Quirrell moment in real life; a massive proportion of people on LW are deferring their entire worldview to obvious supervillains.

• Trevor1 11 Nov 2022 20:34 UTC

  5 points

  0 ∶ 0

  on: In­stru­men­tal con­ver­gence is what makes gen­eral in­tel­li­gence possible

  I think to maximize legibility to different kinds of people, it helps to include “instrumental convergence usually involves all things converging on the path of accumulating resources (including information) and self-preservation, and, by extension, anything that contributes to self-preservation and accumulating resources”.

  Instrumental Convergence might seem like common language, but it certainly is not, and it really should be. Evocative examples increase word count, but they’re so helpful that cutting them out in order to lower word count is basically Goodharting. At minimum, it will help an unfamiliar reader quickly and conveniently explain it to others in many casual conversations.

  Also, this post is very helpful, I have archived it in my open-source intelligence/​research folder and I plan to cite it in the future. The way this post was written is actually the ideal way to explain AI safety to someone for the first time since it allows a wide variety of intelligent people to really dive into the problem in a way they understand. I recommend submitting it to the AI Safety Public Materials Contest which I recently confirmed will still read, evaluate, and implement good papers for explaining AI safety to people for the first time. Regardless of whether they have any money to award, they will still credit the author with a ranking that will build credibility.