Brendon_Wong

Can You Give Sup­port or Feed­back for My Pro­gram to Alle­vi­ate Poverty?

Safety-First Agents/​Ar­chi­tec­tures Are a Promis­ing Path to Safe AGI

• Brendon_Wong 29 Nov 2013 8:25 UTC

  13 points

  on: 2013 Less Wrong Cen­sus/​Survey

  Answered all questions, I hope I helped!

  I’m very curious to see how the monetary reward works out.

• Brendon_Wong 18 Jul 2013 18:21 UTC

  11 points

  on: Wel­come to Less Wrong! (5th thread, March 2013)

  Hello! I’m a 15 year old sophomore in high school, living in the San Francisco Bay Area. I was introduced to rationality and Less Wrong while interning at Leverage Research, which was about a month ago.

  I was given a free copy of Chapters 1-17 of HPMOR during my stay. I was hooked. I finished the whole series in two weeks and made up my mind to try and learn what it would be like being Harry.

  I decided to learn rationality by reading and implementing The Sequences in my daily life. The only problem was, I discovered the length of the Eliezer’s posts from 2006-2010 was around around 10 Harry Potter books. I was told it would take months to read, and some people got lost along the way due to all the dependencies.

  Luckily I am very interested in self improvement, so I decided that I should learn speed reading to avoid spending months dedicated solely to reading The Sequences. After several hours of training, I increased my reading speed (with high comprehension) five times, from around 150 words per minute to 700 words per minute. At that speed, it will take me 33.3 hours to read The Sequences.

  It seems like most people advise reading The Sequences in chronological order in ebook form. Is using this ebook a good way to read The Sequences? Also, If I could spend 5 seconds to a minute after each blog post doing anything, what should I do? I was thinking of making some quick notes for myself to remember everything I read, perhaps with a spaced repetition system, or figuring out all the dependencies to smooth the way for future readers, perhaps leading to the easier creation of a training program...

  Thanks for all your help, and I look forward to contributing to Less Wrong in the future!

  What links here?

  • Brendon_Wong's comment on Wel­come to LessWrong (For high­school­ers) by Curiouskid (18 Jul 2013 18:38 UTC; 1 point)

• Brendon_Wong 19 Jul 2013 6:42 UTC

  8 points

  on: In­stru­men­tal ra­tio­nal­ity/​self help re­sources

  I have my own website and I can set up a wiki/​social network dedicated to self improvement/​instrumental rationality. Is there significant community interest?

  If there is significant interest, would a wiki with social networking features such as forums and chat be more useful, or should I just stick to a wiki?

  Wiki options: DocuWiki, MediaWiki, PmWiki, and WikkaWiki

  Social networking options (with wiki): BuddyPress on WordPress

  I am open to other suggestions, but they may be harder or more expensive for me to set up. If anyone has experience setting up websites, their help would be greatly appreciated (if the website concept is deemed useful).

Pos­i­tive Book and Other Me­dia Recom­men­da­tions for a Teen Audience

• Brendon_Wong 20 Jul 2023 1:24 UTC

  5 points

  2

  in reply to: Roman Leventov’s comment on: In­ter­nal in­de­pen­dent re­view for lan­guage model agent alignment

  I implied the whole spectrum of “LLM alignment”, which I think is better to count as a single “avenue of research” because critiques and feedback in “LMA production time” could as well be applied during pre-training and fine-tuning phases of training (constitutional AI style).

  If I’m understanding correctly, is your point here that you view LLM alignment and LMA alignment as the same? If so, this might be a matter of semantics, but I disagree; I feel like the distinction is similar to ensuring that the people that comprise the government is good (the LLMs in an LMA) versus trying to design a good governmental system itself (e.g. dictatorship, democracy, futarchy, separation of powers, etc.). The two areas are certainly related, and a failure in one can mean a failure in another, but the two areas can involve some very separate and non-associated considerations.

  It’s only reasonable for large AGI labs to ban LMAs completely on top of their APIs (as Connor Leahy suggests)

  Could you point me to where Connor Leahy suggests this? Is it in his podcast?

  or research their safety themselves (as they already started to do, to a degree, with ARC’s evals of GPT-4, for instance)

  To my understanding, the closest ARC Evals gets to LMA-related research is by equipping LLMs with tools to do tasks (similar to ChatGPT plugins), as specified here. I think one of the defining features of an LMA is self-delegation, which doesn’t appear to be happening here. The closest they might’ve gotten was a basic prompt chain.

  I’m mostly pointing these things out because I agree with Ape in the coat and Seth Herd. I don’t think there’s any actual LMA-specific work going on in this space (beyond some preliminary efforts, including my own), and I think there should be. I am pretty confident that LMA-specific work could be a very large research area, and many areas within it would not otherwise be covered with LLM-specific work.

Effec­tive Ra­tion­al­ity Train­ing Online

Notes on the im­por­tance and im­ple­men­ta­tion of safety-first cog­ni­tive ar­chi­tec­tures for AI

• Brendon_Wong 14 Dec 2023 16:32 UTC

  3 points

  0

  on: Some for-profit AI al­ign­ment org ideas

  Great article! Just reached out. A couple ideas I want to mention are working on safer models directly (example: https://​​www.lesswrong.com/​​posts/​​JviYwAk5AfBR7HhEn/​​how-to-control-an-llm-s-behavior-why-my-p-doom-went-down-1), which for smaller models might not be cost prohibitive to make progress on. There’s also building safety-related cognitive architecture components that have commercial uses. For example, world model work (example: https://​​www.lesswrong.com/​​posts/​​nqFS7h8BE6ucTtpoL/​​let-s-buy-out-cyc-for-use-in-agi-interpretability-systems) or memory systems (example: https://​​www.lesswrong.com/​​posts/​​FKE6cAzQxEK4QH9fC/​​qnr-prospects-are-important-for-ai-alignment-research). My work is trying to do a few of these things concurrently (https://​​www.lesswrong.com/​​posts/​​caeXurgTwKDpSG4Nh/​​safety-first-agents-architectures-are-a-promising-path-to).

• Brendon_Wong 7 Aug 2023 6:39 UTC

  3 points

  1

  in reply to: Simon Goldstein’s comment on: Safety-First Agents/​Ar­chi­tec­tures Are a Promis­ing Path to Safe AGI

  Yep, I agree that there’s a significant chance/​risk that alternative AI approaches that aren’t as safe as LMAs are developed, and are more effective than LMAs when run in a standalone manner. I think that SCAs can still be useful in those scenarios though, definitely from a safety perspective, and less clear from a performance perspective.

  For example, those models could still do itemized, sandboxed, and heavily reviewed bits of cognition inside an architecture, even though that’s not necessary for them to achieve what the architecture working towards. Also, this is when we start getting into more advanced safety features, like building symbolic/​neuro-symbolic white box reasoning systems that are interpretable, for the purpose of either controlling cognition or validating the cognition of black box models (Davidad’s proposal involves the latter).

• Brendon_Wong 18 Jul 2023 21:47 UTC

  3 points

  2

  in reply to: Roman Leventov’s comment on: In­ter­nal in­de­pen­dent re­view for lan­guage model agent alignment

  Do you have a source for “Large labs (OpenAI and Anthropic, at least) are pouring at least tens of millions of dollars into this avenue of research?” I think a lot of the current work pertains to LMA alignment, like RLHF, but isn’t LMA alignment per say (I’d make a distinction between aligning the black box models that compose the LMA versus the LMA itself).

• Brendon_Wong 3 May 2023 11:45 UTC

  3 points

  0

  on: CAIS-in­spired ap­proach to­wards safer and more in­ter­pretable AGIs

  Have you seen Seth Herd’s work and the work it references (particularly natural language alignment)? Drexler also has an updated proposal called Open Agencies, which seems to be an updated version of his original CAIS research. It seems like Davidad is working on a complex implementation of open agencies. I will likely work on a significantly simpler implementation. I don’t think any of these designs explicitly propose capping LLMs though, given that they’re non-agentic, transient, etc. by design and thus seem far less risky than agentic models. The proposals mostly focus on avoiding riskier models that are agentic, persistent, etc.

• Brendon_Wong 12 Oct 2014 20:59 UTC

  3 points

  in reply to: Sysice’s comment on: Pos­i­tive Book and Other Me­dia Recom­men­da­tions for a Teen Audience

  Thank you for your help! I have edited my post with additional information. My audience is a general youth audience, think of promoting content to an entire high school, with “average teenagers” and people that might be more interested in content. Of course, some people will be more interested than others, so a wide variety of recommendations for different interest groups is better. I’m primarily looking for books that promote ethical/​altruistic behavior, I’m not sure if any of your beforementioned recommendations do so.

• Brendon_Wong 19 Jul 2013 19:01 UTC

  3 points

  in reply to: alex_zag_al’s comment on: Does an­chor­ing on dead­lines cause pro­cras­ti­na­tion?

  Sorry, that comment did not come out correctly. My bad. What I meant is that you might not have the right reason that explains why moving deadlines closer reduces procrastination.

  I don’t think anchoring is related to procrastination. Making the deadline closer may help procrastination, but it does not seem like anchoring is the mechanism by which it happens.

• Brendon_Wong 4 Dec 2023 9:03 UTC

  2 points

  0

  in reply to: RogerDearnaley’s comment on: How to Con­trol an LLM’s Be­hav­ior (why my P(DOOM) went down)

  I appreciate your thoughtful response! Apologies, in my sleep deprived state, I appear to have hallucinated some challenges I thought appeared in the article. Please disregard everything below “I think some of the downsides mentioned here are easily or realistically surpassable...” except for my point on “many-dimensional labeling.”

  To elaborate, what I was attempting to reference was QNRs which IIRC are just human-interpretable, graph-like embeddings. This could potentially automate the entire labeling flow and solve the “can categories/​labels adequately express everything?” problem.

• Brendon_Wong 28 Apr 2023 3:42 UTC

  2 points

  1

  on: Ca­pa­bil­ities and al­ign­ment of LLM cog­ni­tive architectures

  Have you read Eric Drexler’s work on open agencies and applying open agencies to present-day LLMs? Open agencies seem like progress towards a safer design for current and future cognitive architectures. Drexler’s design touches on some of the aspects you mention in the post, like:

  The system can be coded to both check itself against its goals, and invite human inspection if it judges that it is considering plans or actions that may either violate its ethical goals, change its goals, or remove it from human control.

• Brendon_Wong 30 Jun 2015 8:56 UTC

  2 points

  in reply to: Strangeattractor’s comment on: Can You Give Sup­port or Feed­back for My Pro­gram to Alle­vi­ate Poverty?

  Thank you for your suggestions. I have in fact surveyed people and organizations in Belize. The general consensus is that there are a lot of people who are unemployed or working for very low wages, and getting higher paying employment would improve their standard of living. You mentioned a small scale pilot, we have actually run many such pilots, which is how we found that it would be possible to help people earn around $3 USD an hour. We are currently working on remote testing of our program before actually sending staff to Belize.

• Brendon_Wong 25 Jun 2015 23:56 UTC

  2 points

  in reply to: ChristianKl’s comment on: Can You Give Sup­port or Feed­back for My Pro­gram to Alle­vi­ate Poverty?

  Thanks for your question! This particular project is charitable in nature and would probably require funding to get off the ground and expand more rapidly. Since it is not expected to attract for-profit support, especially because it would probably not be a particularly profitable venture, most funding would probably come from people/​organizations with non-profit motives. People/​organizations with non-profit motives generally only donate to nonprofits, which have a better public image and are more trusted to pursue altruistic goals like donors expect. We can also offer tax-deductible donations to donors, and we do not get taxed on donations or income, which gives us a financial advantage for attracting more donations and for earning more income. The only benefits to starting a for-profit venture I can think of would be greater freedom in compensating supporters and in our operations, but since this project is not expected to be a huge profit maker and has charitable intentions, I did not choose the for-profit path, although of course I could consider it in more detail.

• Brendon_Wong 16 Sep 2013 4:44 UTC

  2 points

  in reply to: John_Maxwell’s comment on: Re­quest for Ad­vice: Un­school or High School?

  Since you asked me about the most effective technique I’ve used since this point, I started using the pomodoro technique with Beeminder. I have experienced a very dramatic increase in productivity. Thank you very much!