This is my impression too.
Why do you think it died?
Chris_Leong
Karma: 8,001
I am pretty worried that as we get closer towards RSI that progress will have some momentum of its own. We might already be past that point.
If the US is a hard no through 2028, much of CeSIA’s strategy is largely deferred until conditions change.
This is a bit of a hot-take, but sounds like you might need a more flexible strategy?
“We do not think that convincing more of the public to be concerned about AI risks is our comparative advantage at the moment. This is both because other organisations are already dedicating significant resources to mass communications and because we think that AI progress itself will be the primary driver of our growth. We benefit from being the largest AI protest organisation and positioning ourselves as focused on the risks of future AI, which naturally funnels people concerned about those risks into our ranks.”
Seems sensible.
I found the post interesting, but I agree with Q Home that the OP’s argument doesn’t quite work.
There’s some tension between the posts “not sufficiently rigorous” and “overly focused on rigor” criticisms, that needs to be handled with not care.
If he wants to criticise a field for formalisms that are disconnected from reality, then criticizing analytical philosophy would be better. If he wants to criticise a field for be handwavey, then a continental comparison would be better. But making the comparison and doing both doesn’t quite work, at least for me.
I feel quite nervous about this plan (primarily “middle powers threatening to sell semiconductor equipment and chips to China instead of the US”).
Navigating the intelligence explosion feels hard enough as is. I’m worried that encouraging middle powers to play both sides will just lead to chaos.
I see where you’re coming from. The US is much less trustworthy than it used to be. I can imagine worlds where this plan would lead to good outcomes. However, my intuition is that the chances of this plan blowing up in our faces is way too high. Maybe, I’d change my view if I spent more time reflecting and digging into the facts, but this is where I am at the moment.
”For it to be truly binding, the US must permanently hand over military and political power to AI.”
Also, this is not something we want to rush. I struggle to see how this could happen fast enough to reassure the middle powers, whilst also being late enough that this wouldn’t be utterly reckless. The best counter I was able to come up with is that being able to counter threats might force a handoff anyway and that there might counterintuitively be no difference in the amount of trust you need for a temporary handoff vs. a permanent handoff given the ability of an AI to transform the former into the later.
Nervous system co-regulation is a fancy technical term that, among other things, refers to the observation that being in the presence of someone else can make you more or less stressed.
I recently visited my friend who has a dog.
It was interesting because there was quite a strong effect where if I was feeling a bit anxious (due to the dog jumping around), the dog would feel anxious, but if I took a few deep breaths and calmed down, then the dog would calm down.
My intuition (not just based on this single experience) is that dogs are more reactive in this sense than humans because humans are often shaped by social norms to appear as though everything is fine.
So training with a dog might be a reasonable way to learn this skill.Regarding Chakras, you might find Scott Alexander’s review of Origin Of Consciousness In The Breakdown Of The Bicameral Mind interesting:
“Jaynes (writing in the 1970s) was both a psychology professor at Princeton and an expert in ancient languages, so the perfect person to make this case. He reviews various samples of Bronze Age writing from before and after this period, and shows that the early writings have no references to mental processes, and the later ones do. When early writings do have references to mental processes, they occur in parts agreed by scholars to be later interpolations. If, with no knowledge of the language itself, you tried to figure out which parts of a heavily-redacted ancient text were early vs. late by their level of reference to mental processes, you could do a pretty decent job.
… Jaynes uses it to trace the development process, showing how older sections of the Iliad treat psychology in different ways than newer ones.So for example, a typical translation might use a phrase like “Fear filled Agamemnon’s mind”. Wrong! There is no word for “mind” in the Iliad, except maybe in the very newest interpolations. The words are things like kardia, noos, phrenes, and thumos, which Jaynes translates as heart, vision/perception, belly, and sympathetic nervous system, respectively. He might translate the sentence about Agamemnon to say something like “Quivering rose in Agamemnon’s belly”. It still means the same thing – Agamemnon is afraid – but it’s how you would talk about it if you didn’t have an idea of “the mind” as the place where mental things happened – you would just notice your belly was quivering more. Later, when the Greeks got theory of mind, they repurposed all these terms. You can still find signs of this today, like how we say “I believe it in my heart”. In fact, you can still find this split use of phrenes, which has survived into English both as the phrenic nerve (a nerve in the belly) and schizophrenia (a mental disease). As the transition wore on, people got more and more flowery about the kind of feelings you could have in your belly or your heart or whatever, until finally belly, heart, and all the others merged into a single Mind where all the mental stuff happened together.”
AI Risk Agility Plans—v0.1
COVID is an interesting example to choose.
The measures only worked because it wasn’t more deadly. Worse, the resources spent countering the pandemic were much more than those required to create such a pandemic.
Further, the government’s actions burned a lot of political capital that might make it harder to respond to previous pandemics.
I doubt the things you’re suggesting would have made much of a difference. People believe what they want to believe.
I’m not sure we want the cab rank rule.
Maybe for certain domains—but for others we basically want bad actors to have their influence dimished.
Fire departments do this rather than the police?
Wow… this is a great post!
My main worry would be that someone with an OCD personality might become obsessed with avoiding “Threat Monitoring! Worrying! Ruminating!” as some kind of counter-obsession (I would go stronger and say that we should expect this as the default).
Which would almost certainly be a significant improvement. However, our brains did evolve these behaviours for a reason, so there’s reason to be wary.
Thanks, I’ll check it out!
This ignores the offence-defence balance which, in some circumstances, may massively benefit the attacker.
I like this.
Feels very aligned with the Friendly Gradient Hacker proposal. It certainly carries risks, but I’m starting to suspect that we will be forced to go down this path.
Fascinating post. I appreciate how it challenges conventional wisdom and I’ll have to spend more time thinking these points through.
One thing that confused me though is that this is exactly the kind of post I would haved imagined someone writing if they were trying to defend the Anthropic bet and my understanding was that you were opposed to this?
Interesting. Thanks for explaining.
Just thought I’d share a post I wrote about the potential promise of a “wisdom explosion” in case that’s of interested to you—I’m unsure, but I see some potential resonance/synergy with your perspective—https://aiimpacts.org/some-preliminary-notes-on-the-promise-of-a-wisdom-explosion/ .
Could you explain AI reform? I didn’t quite understand your description.
Gary Tan and Marc Andreessen were boosting it at one point.