wunan

• wunan 20 Aug 2022 21:17 UTC

  57 points

  38

  on: What’s the Least Im­pres­sive Thing GPT-4 Won’t be Able to Do

  There are probably less impressive things than this that it won’t be able to do, but here’s one prediction in which I am about 90% confident:

  If you invent or find a board game of similar complexity to chess that GPT-4 has never seen before and explain the rules using only text (and, if GPT-4 is multimodal, also images), GPT-4 will not be able to perform as well at the game as an average human who has never seen the game before and is learning it for the first time in the same way. I say “perform as well as” rather than “beat” because, depending on the specific rules of the game, I expect GPT-4 will likely not output legal moves.

  This prediction applies to whatever OpenAI reveals with the name GPT-4 or that is clearly the GPT-3 successor regardless of size, assuming it’s revealed in the next two years and is not trained on specifically that task (e.g. by generating tens of thousands of unique synthetic board game examples with synthetic dialogues where they’re explained and played over text).

• wunan 18 Oct 2021 0:16 UTC

  34 points

  in reply to: Viliam’s comment on: My ex­pe­rience at and around MIRI and CFAR (in­spired by Zoe Curzi’s writeup of ex­pe­riences at Lev­er­age)

  On one hand, meditation—when done without all the baggage, hypothetically—seems like a useful tool. On the other hand, it simply invites all that baggage, because that is in the books, in the practicing communities, etc.

  I think meditation should be treated similarly to psychedelics—even for meditators who don’t think of it in terms of anything supernatural, it can still have very large and unpredictable effects on the mind. The more extreme the style of meditation (e.g. silent retreats), the more likely this sort of thing is.

  Any subgroups heavily using meditation seem likely to have the same problems as the ones Eliezer identified for psychedelics/​woo/​supernaturalism.

[Question] Avoid­ing “en­light­en­ment” ex­pe­riences while med­i­tat­ing for anx­iety?

OpenAI charter

• wunan 26 Aug 2022 16:04 UTC

  15 points

  0

  in reply to: Omid’s comment on: What’s the Least Im­pres­sive Thing GPT-4 Won’t be Able to Do

  I tried giving this to GPT-3 and at first it would only give the tautological “pawns become more powerful” example, then I expanded the prompt to explain why that is not a valid answer, and it gave a much better response.

  I believe this response is the same as your fourth bullet point example of a good answer.

  

  Here’s the prompt in copy/​pastable format for anyone who wants to try playing with it:

  Consider a new variant of chess, in which each pawn can move up to six squares forward on its first move, instead of being limited to one or two squares. All other rules remain intact. Explain how game balance and strategy is changed with this new variant of chess.

  Your response should share something not immediately obvious about this variant and provide a plausible justification for why it might be true. Some responses that would not succeed would be

  The pawns become more powerful. (Too simple, close to a tautology.)
  New strategies will need to be developed. (Too vague.)
  Bishops become more valuable. (Needs a justification for why we should expect this.)

  Response:

• wunan 11 Nov 2021 16:27 UTC

  LW: 14 AF: 8

  AF

  in reply to: adamShimi’s comment on: Dis­cus­sion with Eliezer Yud­kowsky on AGI interventions

  Context for anyone who’s not aware:

  Nerd sniping is a slang term that describes a particularly interesting problem that is presented to a nerd, often a physicist, tech geek or mathematician. The nerd stops all activity to devote attention to solving the problem, often at his or her own peril

  Here’s the xkcd comic which coined the term.

• wunan 2 Jun 2022 19:59 UTC

  13 points

  in reply to: supposedlyfun’s comment on: sup­pos­ed­lyfun’s Shortform

  Sci-hub lets you get around paywalls for pretty much all academic articles.

• wunan 2 Mar 2018 13:17 UTC

  11 points

  on: Begin­ner’s Med­i­ta­tion for the No Bul­lshit Individual

  I don’t think this guide goes into enough detail. I had read instructions many times that were essentially the same as this and attempted them consistently every day for weeks or months and made very little progress in terms of improving my attention. There was very little difference from if I had been following the instructions “just sit and relax for 15 minutes.”

  In my experience, the problem isn’t that meditation is often treated as something that’s too complex when actually it’s very simple, it’s that it’s treated as something very simple when actually it’s pretty complex.

  What did help was reading “The Mind Illuminated,” which breaks things down into much more detail. Attempting meditation with the instructions in that book was a very different experience from my earlier attempts. There was a very noticeable improvement in my ability to intentionally maintain my attention within the first few sessions. In fact I made such rapid progress that I stopped after a week because I wanted to take some time to reassess whether this was really a path I wanted to go down (the book provides instructions all the way to “awakening” or “enlightenment”). I’m currently still assessing.

[Question] COVID con­ta­gious­ness af­ter nega­tive tests?

• wunan 2 Oct 2022 15:25 UTC

  10 points

  3

  on: Paper: Large Lan­guage Models Can Self-im­prove [Linkpost]

  Another similar result was that AlphaFold was trained on its own high-confidence predictions for protein sequences with unknown structures:

  The AlphaFold architecture is able to train to high accuracy using only supervised learning on PDB data, but we are able to enhance accuracy (Fig. 4a) using an approach similar to noisy student self-distillation³⁵. In this procedure, we use a trained network to predict the structure of around 350,000 diverse sequences from Uniclust30³⁶ and make a new dataset of predicted structures filtered to a high-confidence subset. We then train the same architecture again from scratch using a mixture of PDB data and this new dataset of predicted structures as the training data, in which the various training data augmentations such as cropping and MSA subsampling make it challenging for the network to recapitulate the previously predicted structures. This self-distillation procedure makes effective use of the unlabelled sequence data and considerably improves the accuracy of the resulting network.

  What links here?

  • aogara's comment on Paper: Large Lan­guage Models Can Self-im­prove [Linkpost] by Evan R. Murphy (2 Oct 2022 17:43 UTC; 18 points)

• wunan 1 Aug 2022 23:41 UTC

  9 points

  2

  on: Is there any writ­ing about prompt en­g­ineer­ing for hu­mans?

  As another example, I remember somebody (maybe EY or Gwern?) suggest that prompting people with “how would somebody smarter than you solve this problem?” can actually be effective at soliciting better solutions than just asking them to solve it directly.

  I don’t remember where I saw this so if anybody has a link feel free to share.

• wunan 9 Sep 2020 10:38 UTC

  9 points

  in reply to: Dagon’s comment on: Es­ca­la­tion Out­side the System

  People are literally looting businesses and NPR is publishing interviews supporting it. They’re not just interviewing people who support it—the interviewer also supports it. What makes you think these aren’t actual policy proposals?

  They may only propose it for deep social-signalling reasons as you say, but that doesn’t mean it’s not actually a proposal. Historically, we’ve seen that people are willing to go through with mass murders.

• wunan 29 Jan 2018 19:40 UTC

  9 points

  in reply to: Raemon’s comment on: A LessWrong Crypto Autopsy

  Do we have any data on how well other people who had similar levels of interest in tech did?

• wunan 20 Apr 2021 4:41 UTC

  8 points

  on: [Let­ter] Ad­vice for High School

  To any high schoolers reading this: If I could send just one of the items from the above list back to myself in high school, it would be “lift weights.” Starting Strength is a good intro.

• wunan 16 Jan 2021 17:57 UTC

  8 points

  on: Why Pro­duc­tivity Sys­tems Don’t Stick

  Has this new congruency-based approach led to less, the same, or more productivity than what you were doing before and how long have you been doing it?

• wunan 28 Jul 2020 13:54 UTC

  LW: 8 AF: 4

  AF

  in reply to: Andy Jones’s comment on: Are we in an AI over­hang?

  In the Gwern quote, what does “Even the dates are more or less correct!” refer to? Which dates were predicted for what?

• wunan 2 Mar 2018 18:23 UTC

  8 points

  in reply to: moridinamael’s comment on: Begin­ner’s Med­i­ta­tion for the No Bul­lshit Individual

  It’s analogous to a music teacher instructing you to just sit down and play some notes, any notes, for twenty minutes. It would be amazing if you made progress that way.

  This is exactly how it felt for me—I even remember thinking this exact same metaphor after practicing TMI and reflecting on the difference between it and my previous attempts.

• wunan 11 Nov 2021 15:19 UTC

  7 points

  in reply to: Adam Zerner’s comment on: Dis­cus­sion with Eliezer Yud­kowsky on AGI interventions

  If MIRI hasn’t already, it seems to me like it’d be a good idea to try reaching out. It also seems worth being at least a little bit strategic about it as opposed to, say, a cold email.

  +1 especially to this—surely MIRI or a similar x-risk org could attain a warm introduction with potential top researchers through their network from someone who is willing to vouch for them.

• wunan 25 Aug 2021 21:37 UTC

  7 points

  on: How to turn money into AI safety?

  Of the bottlenecks I listed above, I am going to mostly ignore talent. IMO, talented people aren’t the bottleneck right now, and the other problems we have are more interesting.

  Can you clarify what you mean by this? I see two main possibilities for what you might mean:

  • There are many talented people who want to work on AI alignment, but are doing something else instead.

  • There are many talented people working on AI alignment, but they’re not very productive.

  If you mean the first one, I think it would be worth it to survey people who are interested in AI alignment but are currently doing something else—ask each of them, why aren’t they working on AI alignment? Have they ever applied for a grant or job in the area? If not, why not? Is money a big concern, such that if it were more freely available they’d start working on AI alignment independently? Or is it that they’d want to join an existing org, but open positions are too scarce?

• wunan 22 Apr 2020 22:47 UTC

  7 points

  on: My ex­pe­rience with the “ra­tio­nal­ist un­canny valley”

  It might help if you try to think less in terms of making rationality and EA part of your identity and instead just look at them as some things you’re interested in. You could pursue the things you’re interested in and become a more capable person even if you never read anything else from the rationality community again. Maybe reading stuff from people who have achieved great things and had great ideas and who have not been influenced by the rationality community (which, by the way, describes most people who have achieved great things and had great ideas) would help? E.g. Paul Graham’s essays are good (he’s kind of LW-adjacent, but was writing essays long before the rationality community was a thing): http://​​paulgraham.com/​​articles.html

  I think the rationality community is great, it has hugely influenced me, and I’m glad I found it, but I’m pretty sure I’d be doing great stuff even if I never found it.