Sure, but you get feedback for whether it helps customers with their immediate problems. You don’t get feedback on whether it helps with AI safety.
It’s the direction vs speed thing again. You’ll get good at building widgets that sell. You won’t get good at AI notkilleveryoneism.
Such a good post! I think it’d be be received well on the EA Forum too. Obviously applicable to EA in a million different ways.
Can’t tell you how many times I’ve had to stare into the abyss of a crucial consideration or study result showing that maybe the thing I’d been working on didn’t help or was even maybe net negative.
Don’t know much about accumulated heavy metals, but they’re really low on the food chain, so they’re a priori going to have less of those than those higher up the food chain.
You can see their nutritional profile here. Sky high in B12, great at omega-3s and iron.
I also predict they’ll be good at a wide variety of things we don’t know we need yet, since they’re as close to a “whole food” as you can get. You’re eating almost the whole animal, instead of just a part of it.
Thank you for writing this! Gave me a great concept I’m going to use going forward.
The Grok suggesting organ harvesting of illegals surprised me, so I tried to replicate it.
It did not replicate.
It gave really reasonable answers, including trying to help countries of origin to be more prosperous and stable and making merit-based paths to legal status.
I mean, it still said stuff like using covert operations to stabilize countries, which is pretty dark arts, but very far away from organ harvesting and public executions.
I think that when robotics becomes sufficiently anthropomorphic the AI backlash will really come into full swing.
Imagine Sydney Bing threatening users but it’s a robot in your house.
The visceral reaction is going to be way stronger than all the papers we could publish.
Agreed about populism. Populism is “us the pure majority against the corrupt elites”, which can apply to all sorts of ideologies.
There’s right wing populism (e.g. “us the poor working class against the corrupt East coast elites”) and left wing populism (e.g. “us the poor majority against the corrupt corporate giants”, think the Occupy movement). There’s also populism in Latin America that’s more focused on fighting political clientelism, etc.
I think you could easily change the article to say “MAGA” instead of “populism” and get the same point across, though it’s not that big of a deal.
“The dominant ideologies are not necessarily those with the best ideas—they are those with the best survival strategies.”
I’d add that it’s: idologies that survive and reproduce.
In your framework, spreading=reproducing.
Self-sealing and retention=surviving.
If an idea is open to revision or people changing their minds, it will likely “die” quickly. It needs to spread and then have self-protection mechanisms so the ideology doesn’t die when exposed to contradictory ideas.
Deep Research was this for me, at first. Some of its summaries were just pleasant to read, they felt so information-dense and intelligent! Not like typical AI slop at all! But then it turned out most of it was just AI slop underneath anyway,
Can you elaborate on what you mean by this? Do you mean it’s hallucinating a ton underneath? Or that the writing is somehow bad? Or something else?
I provided one here!
Totally agree about LLMs. I’ve recently been finding giving o3 Deep Research this prompt to be super helpful:
“X=[thing I want to research]
Do a deep dive into X. Tell me the steelman arguments in favor of X.
Then tell me the steelman counter-arguments to X.
Then tell me the steelman counter-counter-arguments to X.
Then tell me the steelman counter-counter-counter-arguments to X
Make sure to link to primary sources and the full thing that happened, to avoid things being quoted out of context”
It’s been particularly helpful for investigating anything political.
To be fair, I think you could make the case that the technological advancement of the Western European powers led to almost a DSA.
England had the biggest empire in the history of the world (population wise). The countries which were technologically ahead of others did kinda stomp on the rest of the world
It wasn’t literally one world power, but it was a sizeable fraction that way, especially if you consider European culture as a unit instead of just the UK.
Ah well. At least you can take credit for the name then.
Fair enough. There are some for profits where profit and impact are more related than others.
But it’s also quite likely your evals are not actually evaluating anything to do with x-risks or s-risks, and so it just feels like it’s making progress, but isn’t.
I’m assuming here people are trying to prevent AI from killing everyone. If you have other goals, this doesn’t apply.
I’d say this is the same thing for AI for-profits from the perspective of AI notkilleveryoneism. Probably, the modal outcome is slightly increasing the odds AI kills everyone. At least the non-profits the modal outcome is not doing anything, rather than making things worse.