China is a big one for this. Large numbers of experts, including many non-pundits, have been widely forecasting China’s imminent demise on 3 separate occasions over the last 1.5 years (disclaimer: if you look closely you’ll see that many of them were being vague and had way too much plausible deniability to consider it a “forecast”, and I only used the word “forecast” above because many were vaguely acting like they were forecasting).
Trevor1
Karma: 1,007
Is this the first time that the word “Boltzmann” has been used to describe contemporary/near future ML? If not, how frequently has the word “boltzmann” been used in this way?
Also, I know this question might be a bit of a curve ball, but what pros and cons can you think of for using the word “boltzmann”? (feel free to DM me if there’s anything you’d rather not say publicly, which is definitely the right way to approach it imo). I’m really interested in AI safety communication, which is why I’m asking these slightly off-topic questions.
I think this would go down more smoothly with a few more examples of level 4. I found the “Level 4: A trial by ordeal or trial by combat lacks and denies the concept of justice entirely” pretty helpful for describing lizard brain/association diverging from reality, but still feeling correct enough for a person to choose it.
I do AI policy, and it has been extremely rare for me to see something that sticks out to me as something as valuable as this. I don’t think you’re aware how incredibly valuable your gifs are for describing problems with RLHF. The human brain is structured to glaze over, ignore, or forget text, while considering visual evidence with the full force of reasoning faculties. Visual information is vastly more intuitive and digestible that written language, it is like “getting your foot in the door” to someone’s limited daily allowance of attention and higher reasoning faculties. Although it requires some buy in to figure out ways to correctly and honestly describe the problem with gifs, the communication value (to other ML researchers) is massive and the ratio of honest, successful communication to effort is extremely high. Using gifs to visually describe the problems with RLHF just scales really well with more time, effort, and cognition spent on really accurate gifs, after making the first gifs.
Most people don’t notice or seriously consider the problem with RLHF because they don’t feel like digesting text criticizing RLHF, and more gifs will give experts a fair chance to have accurate thoughts about the problems with RLHF. I’m not an expert and I don’t know how difficult it is to actually make gifs that can describe the complex problems, but I do know that if the bare minimum is managed at making such gifs, the paper has a much higher chance of causing a paradigm shift among ML researchers than the average ML researcher might think. Even if describing most of the RLHF problems with gifs is impossible or systematically fails, it will still have its effect amplified by allowing ML researchers to begin communicating problems to tech executives and policymakers who manage their projects and funding.
I hope that this isn’t your final compendium on RLHF (I’m bookmarking it either way) and that you spend at least a little bit of time evaluating whether it’s possible to describe problems with RLHF to ML researchers using gifs. This is a gold mine, and it never occurred to me that it could be done until I saw that gif. If you can’t figure out a way to do it yourself, I recommend asking around for information about funding such as the EA future fund and ask about funds and grantmaking at rationalist events and people will probably be able to connect you to large sources of funding, teams of artists and animators to handle the grunt work, or even other ML researchers who have a lot of knowledge of ways to figure out ways to accurately depict RLHF problems visually (think rob bensinger or eliezer yudkowsky). I hope that this isn’t your final compendium on RLHF (I’m bookmarking it either way) and that you spend at least a little bit of time evaluating whether it’s possible to describe problems with RLHF to ML researchers using gifs.
I found this post extremely helpful.
However, there’s two things I’m suspicious about. The first was Brave Search and Kagi; facebook bends over backwards to avoid giving users that kind of control, and this is a norm I’ve observed throughout web services run by large corporations. Kagi and Brave Search don’t seem to have ties to a large corporation, but I’m still suspicious that something that good could exist in the modern era. I’d love to be proven wrong about this.
The other thing is NYT wirecutter. I remember some pretty suspicious content there, although I never wrote anything down. It was never anywhere near as serious as that one time WSJ wrote an outright propaganda piece praising Roblox for hijacking the minds of millions of 10-year-olds. Maybe NYT innocently followed some trends that were actually started by corrupted reviewers, and that set off my warning bells because I only ever observed wirecutter.
I definitely want to see more of this.
I think you have a pretty solid understanding of this, and what would have been really helpful would be if there was links or sources. Not because I doubt any of the claims here, but because there’s a pretty long list of things here that I’d like to read more about, but there’s nothing to click.
This is a big part of Open Source Intelligence which is becoming more popular on Lesswrong; even if someone gets half of the stuff wrong or doesn’t know what they’re talking about (which isn’t what happened here, this is a solid understanding), there’s so much going on in the international space that providing links gives any reader tons of opportunities to discover tons of information, that they probably never would have collided with otherwise.
Can you go into more detail about China? I’ve heard that China is the biggest concern since there’s a big AI gap between China and the rest, and I’ve also heard that there’s a pretty big and potentially long-lasting gap between China’s AI industry and the US due to the US successfully recruiting talent from China (of course, all these countries have intelligence agencies that can steal secrets from all sorts of places). But it’s really hard to get good info about China because, as you said, it’s a really non-transparent country.
I really liked the “aside” on this.
I think it’s also worth noting that WaPo is a de-facto Amazon subsidiary (technically 100% owned by Bezos, Amazon’s chairman). I’m not sure what the implications are here, just that it’s worth noting.
If the republicans are bringing out the accusations of exploited taxpayers, then you know they’re serious.
(disclaimer: this doesn’t actually tell you how serious they are, the only way to track the seriousness of anything related to congress is to follow the money).
I’ve seen some posts on AI timelines. Is this the most definitive research on AI timelines? Where is the most definitive research on AI timelines? Does the MIRI team have a post that gives dates? I’m at an important career branch so I need solid info.
I’m sorry that this is all I have to contribute, I don’t know anything about the duration of contagiousness but I still have some potentially decision-relevant factors.
The rapid antigen tests have a history of really high false negativity rates: https://www.lesswrong.com/posts/T4H7w6BqB6mnm9JYq/how-much-bayesian-evidence-from-rapid-antigen-and-pcr-tests but this doesn’t mean they are sensitive in a trustworthy way, it’s more of an indicator that the people making them aren’t trustworthy people.
You want to use p100 masks, not n95 masks (they are available at large hardware stores and Amazon). My model since last winter is that the current strain infects most people who aren’t wearing p100 masks when they go to shared indoor spaces, although the herd immunity since then might mean that it’s infecting closer to 10-50% of the country (US) instead of 95-99%.
That actually goes a long way towards answering the question. This means that in order for it to be connected, the lawsuit would have been on the backburner and the OpenAI-MSFT partnership somehow was either the straw that broke the camel’s back, or it mostly-by-itself triggered a lawsuit that was held in reserve against google. Highly relevant info either way, thank you.
Have you read about the Orthogonality hypothesis? it seems to me like you’d find it interesting. Lots of people think that arbital is the best source, but I like the Lesswrong version better since it keeps things simple and, by-default, includes top-rated material on it.
There’s also Instrumental convergence which is also considered a core concept for Lesswrong (LW version).
Nostal also wrote a pretty good post on some reasons why AI could be extremely bad by default. I definitely disagree that AI would be guaranteed to be good by default, but I do international affairs/China research for a living and I don’t consider myself a very good philosopher, so I myself can’t say for sure because it’s not my area of expertise.
Of course it’s also important to note that these are interviews, and in general, most people in business see press interviews as a cheap way to ease public doubts without actually doing anything (i.e. talk is cheap). This is extremely common in Congress.
I think it’s worth noting that, if anyone is setting up a database of information about world modelling and AI policy (public or private), this post should probably be in that database.
I don’t know how tractable this particular problem is, a really big part of it is that a social media platform can easily tell which articles and which news outlets correlate with a user giving up on the platform for good (or at least spending fewer hours on it), and then shows users fewer things like that in the long run. Superior versions of this article would be the first thing to go, and that’s with the tech level that social media platforms had a decade ago.
This results in the kind of optimization pressure that possibly no human is aware of (even with 2013 ML), let alone the journalist themself. But the all/most of the news articles that result can easily end up being pretty bad at getting people off of social media. The only solution I can think of seems to just be more lesswrong and less social media.
Only one near-vegan out of 5 had solidly good ferritin levels. As I discuss here, that’s a very big deal, potentially costing them half a standard deviation on multiple cognitive metrics.
There’s no control group, so I can’t prove that this is a veganism problem. But I’m quite suspicious.
I might have noticed lower iron levels on the nutrition facts of vegetables, as well as all other vitamins and minerals (relative to a few years ago, but I wasn’t paying much attention a few years ago). Like, a 1-cup serving of organic peas having 6% iron, 4% vitamin B, and 1-0% on everything else, which was weirdly low for basically half a day’s intake of vegetables (this was ~november-december, all vegetables I could find at Whole Foods and Trader Joes in Foggy Bottom DC). It might have had something to do with FDA labelling though. I looked it up and the internet had surprisingly little to say other than that soil/fertilizer quality is degrading. It might plausibly be worth doing an experiment.
They also picked the Los Alamos site for the Manhattan project because it was remote (i.e. far away from potential Japanese or German bombardment) and the view of the nearby mountain range would invigorate the scientists. The military built an entire town there for that, with the schools and centrifuges and sewage system and everything.
Hi Derikk! I’m a huge fan of Cavendish Labs and I’m proud to say that I liked it before it was cool. This is a big step towards doubling the number of full-time alignment researchers currently working on earth. Plus, if something happens to Dustin Moscovitz just like all the other billionaires, I can see Cavendish VT becoming one of the main AI safety hubs of the world.
What are rent prices like? Is there an apartment complex nearby, or is that redundant because can you just outright rent a house for $1000 a month or something?
we found it to be a fantastic place to live, think, and work. Each season brings with it a new kind of beauty to the hills. There are no barriers to a relaxing walk in the woods. There’s practically no light pollution, so the cosmos is waiting outside the door whenever you need inspiration.
I’m based in the DC area, and I’m definitely taking that train for my vacations (iff I’m invited, of course). NYC, Boston, DC, and SF all have traffic, pollution, and noisy hotels.
fall 2021: banking system collapse due to evergrande
spring 2022: covid overruns healthcare system, causing mass graves and widespread system failure
winter 2022: covid overruns healthcare system, causing mass graves and widespread system failure
To be fair, I was a little too harsh on the third time, since many people and most china watchers were very fatigued by then with the ubiquitous plausibly deniable claims of imminent catastrophe in China which didn’t come to pass at all the first two times. Then when there actually were videos popping up of piles of bodybags, everyone got confused. The first two times were pretty strong evidence that experts of all kinds were popping up all over the place making way overconfident predictions of China’s imminent demise. I don’t know whether those predictions persuaded many intelligence agencies or wall street analyst firms, but many people in open source intelligence got bamboozled and started wising up by the third time. I heard that the worst of it was on twitter (alot of respected china watchers apparently tweet), but I’ve never used or trusted twitter so I’m not the right person to ask about how bad it got there.