Economist.
Sherrinford
Karma: 1,315
With my original comment, I am saying that doing something once is not always extremely helpful for doing it later times. I am not sure whether Covid helped learning/preparing for a next pandemic, but I think it had some negative effects for that with respect to the attitudes of the population, and positive effects with respect to scientific insights. I am much less sure whether it applies to an AI pause!
I am skeptical whether the word “trial run” is a good word for a real-world AI pause or pandemic measures starting in 2020. A trial run is a preliminary test, but with real-life anti-pandemic measures there are many things you cannot control, so the learning is limited.
Whether “doing something once” is “extremely helpful for doing it later times” depends on whether people agree—while it is done and afterwards—on whether the measure is/was good, effective, fair, and whether people think the benefits exceeded the costs. It seems relevant whether people believe that it worked well and whether people more or less understood what worked and what didn’t.
In the abstract, I assume that people agreed before 2020 that anti-pandemic measures can be necessary. When the pandemic came and they were enacted, there was less consensus. Additionally, more people seem to evaluate the measures negatively in retrospect. (Personal impression, not a data-based statement.)
During Covid, there was political opposition to the measures, but some years later, this opposition has probably had more time to coordinate and develop.
Thanks, this is a great list!
The only thing I would love to see changed: If there is LessWrong post for an entry, then you should link to that, because there are sometimes helpful comments (okay, I am not neutral here).
I totally agree that “woo” is an insufficient category. It does not even have the full status of “I know it when I see it”, with all the advantages and disadvantages, because I am not always completely sure about each individual practice.
There are several properties that I find worrying:
a practice that does not work as it is explicitly claimed to work,
when a practice only works because of an atmosphere of authority and “letting go” while participants make themselves vulnerable,
strong reliance on selection bias (a technique “works” for some people or they claim that it does, zero effects or harmful effects on others are ignored, the first group are the testimonials for later recruitments),
no real answers to criticisms of all that, or: answering but making up rationalizations for why to use the pratice anyway.
For these points, I would again probably find counterexamples (that is, some practice seems justified even though it has some properties like that), or different opinions on whether some practice fulfills it.
I disagree with your (implicit) suggestion that it is per se inadequate to describe practices as woo if they are taught at CFAR workshops. It is possible that there are questionable “rationality techniques” out there. I think Circling is a candidate for that. If you know a good objective discussion of Circling (or a good comprehensive description of it), I am grateful for a link.
By the way, I like group singing. It nonetheless may have effects like lowering your level of critical thinking. There is no harm in mentioning that.
It is remarkable that in rationalist circles there seems to be a strong demand for woo techniques, rituals and behavior, reinterpreted in some way or emphasizing the possibly useful aspects, but overlooking how they switch off your critical thinking.
Hannah Ritchie has published an article including the per-capita comparisons:
https://hannahritchie.substack.com/p/heat-guns-america-europe
We can’t pause on a dime at the precise second that ‘we’ decide it is important to—pulling the breaks will take a while, during which time we will continue to rocket into danger.
I would generalize this because some people say things like “bad regulation can easily make things worse”, and though it is not clear to me what the author of the post I link to means with that warning, I note that it would be an unrealistic expectation that we end up with exactly the right amount of regulation. Some people warn against AI regulation by referring to nuclear regulation. But it may be necessary to balance the possibilities of “bad overregulation” vs “bad underregulation”, both within a set of realistic policies.
“Doing something once is in general extremely helpful for doing it later times” etc
It depends. Given the politics of anti-pandemic measures, I am not sure that we are as “prepared” against a pandemic as in 2020.
A common refrain among safety-conscious AI developers: “it doesn’t matter if we stop building dangerous AI, because someone else will just build it instead.” Is that really true, though? If a multi-hundred-billion-dollar company comes out and says “We’ve concluded that our product is horribly dangerous, nobody knows how to make it safe, and there’s too high a risk that it leads to human extinction”, this won’t raise any eyebrows? This has no chance of spurring policy-makers into action?
This is not so clear.
Many things should “raise eyebrows”. Hundreds of AI experts have signed statements warning of existential risks. Companies are announcing the development of AGI—companies that ought to be viewed as credible, given that they are already creating technologies that would have been dismissed as unrealistic science fiction not so long ago. Governments talk about being in an AI race, without a real discussion of without truly discussing the finishing line of this race is and what happens when it is reached. All of this seems to take a very minor role of public debate. The Serious People would possibly dismiss any really costly AI safety steps by a company as an obvious sign of madness. The press would be outraged that a company worth so much money was ruining its investors and the economy by abandoning its business model. For any other reaction to emerge in the public debate, different ways of thinking would likely need to be more widespread first. In my understanding, that was one of the reasons why Eliezer started LessWrong.
True. Additionally, there is a weird fallacy:
A) our diets evolved together with our bodies over a very long time.
B) eating whatever you like from the things that are offered in a modern supermarket is healthy (or at least healthier than a standard vegetariqn diet).
Yet B does not follow from A.
It could also be true that
Eating meat was fitness-maximizing under conditions of hunter-gatherer food scarcity, but it is unnecessary today. (Similar to bacalao or pemmican, which may both be useful under certain conditions, but no necessary part of a modern diet. )
People who just eat what they like eat unhealthy stuff and quantities.
An example:
Socrates earns money by offering services as an editor via internet contact. This seems to be a aervice still only done by humans, and editing and replying to the customers takes Socrates more than, well, a minute.
In general, the minds of Socrates seem to think and communicate more or less with human speed. Their advantages mostly seem to be: internet access, perfect memory, no sleep.
A thought as a reaction to a discussion on veganism of the Bayesian Conspiracy podcast:
It does not seem right that being healthy as a typical omnivore is costless wheras being healthy as a vegetarian is very costly at least in terms of doing your research. If you want spend no thoughts on your food and have maximum convenience, that is not the same as taking health into account.
Currently reading Crystal Society, written by Max Harms in 2016… and about to stop reading because the combination of what an AI can cannot do in that book seems so outdated and weird. Good reminder that the world of 2026 was really hard to imagine in 2016.
It is a feature once “you meet the karma thresholds (50 on Personal blogposts, 2000 on Frontpage posts)”. Maybe that threshold is too demanding?
I don’t understand in which way you think it is better to ban Said globally. Is the reason that then nobody has to read his opinion on being banned banned anymore, which he writes somewhere else?
The OP pointed out that you can just personally ban people. Does “So I clear the area of snipers when I can.” mean that you think people are unable to do that when it is good for themselves?
Okay, but then what? Do they all (US companies + US government) believe that—once they have actually built AGSI—there will be a static endpoint at which all their competitors say: “Okay, you’ve invented AGSI now; we all give up”? If so: In what competition, exactly, would they be giving up? Would the rest of the world then stop conducting further AI research? Or would the OpenAI/Anthropic/Google/Musk AI then be deployed to sabotage AI research in the rest of the world? I don’t understand the scenario.
“The US government and leading AI companies have already jointly decided to race hard and win.”
Have they ever defined what exactly this race is? What does its finishing line, or the post-race state of the world look like? What are they trying to accomplish?
My understanding of some things in AI development, which is probably completely wrong:
OpenAI wants to automate AI research.
This is bad for AI researchers, including those working at OpenAI, unless they own shares in the company.
The decision-makers have wriggled out of the non-profit status.
This could potentially mean that they are inclined and have the ability to evade other obligations as well.
If I were an AI developer at OpenAI without a deeper understanding of company structures, corporate law, etc., I would wonder if the decision-makers there could find a way to ensure that they ultimately pocket all the money and I get nothing, even if I currently own shares in the company.
Did you migrate away, and what is your preferred alternative?