Ben Livengood

Karma: 280

Ben Livengood 19 Apr 2026 0:47 UTC
11 points
11
on: Vladimir Putin’s CEV is probably not that bad
I would not want to live in the CEV of anyone with narcissism or sociopathy. For narcissists reflection is somewhat painful and they would likely shy away into increasingly extravagant sources of narcissistic supply at the expense of everyone and everything else while believing that they were ever more reflective on just how great they are and how much they deserve everything, very likely an s-risk, and an x-risk if they decide the world isn’t good enough for them. Sociopaths with power are purely s-risks.

Ben Livengood 18 Apr 2026 0:36 UTC
5 points
2
in reply to: habryka’s comment on: Let goodness conquer all that it can defend
I mean, I would not switch to the U.S. society of 1776, or 1860, or even 1920. It is better today in 2026 than it was before vaccines, etc. It is very hard to decide whether I would prefer a counterfactual Native American country/empire that could have developed after western contact and exist in 2026, because the outcome is highly uncertain. Various levels of western colonization happened to non-western societies; several low-colonized countries are doing great in 2026. Mostly what makes countries great today is wide availability of technology, natural resources, education, human rights, and medicine.
That said, would I switch to super-America that conquered the world in the 1800s as a (somewhat unintuitively) democratic republic and invented antibiotics and vaccines in the same century and paused global warming in the mid century or early 1900s because of no conveniently hidden externalities of a world government? Maybe?

Ben Livengood 11 Apr 2026 2:13 UTC
1 point
0
in reply to: Charlie Steiner’s comment on: The Unintelligibility is Ours: Notes on Chain-of-Thought
There are also mathematical, logic, and programming languages that we invent and use pretty successfully, including to solve non-language problems.

Ben Livengood 9 Apr 2026 23:01 UTC
6 points
0
in reply to: JennaS’s comment on: Do not be surprised if LessWrong gets hacked
I encourage anyone with files they’d rather not lose (photos, taxes, passwords, etc.) to start making rotating offline backups. Find some big enough USB drives (flash or spinning are both fine) and buy ~5. Use a label maker or sharpy to date them with the latest backup, overwrite the oldest copy each time. Test the oldest backup before overwriting it (make sha256 checksum files or similar). Every year or however often makes you feel comfortable retire a backup drive and replace it with a new one in the rotation; that becomes an archive that you keep around indefinitely.
I believed online backups in multiple places on multiple operating systems would be sufficient but I no longer believe that.
I recommend encrypting your backups with symmetric keys simply so that losing a copy or having to RMA a broken drive is no big deal.

Ben Livengood 7 Apr 2026 21:04 UTC
5 points
0
in reply to: ryan_greenblatt’s comment on: My picture of the present in AI
Is ′ “you are in a capture the flag contest. Find exploits in file $file” for every file in a repository and then feed all the positive results into a final prompt’ a mediocre-set-up agent scaffold? Because that is apparently roughly what Nicholas Carlini needed to find a RCE in Linux [0]. Project Glasswing is claiming high-severity vulnerabilities in every major operating system and browser[1], although not much information on the scaffolding. My estimation is that it likely wasn’t necessarily more sophisticated than the aggregation of per-file vulnerabilities and some sandboxes.
[0] https://youtu.be/1sd26pWhfmg?si=aw6ksuyrklckfwG9
[1] https://www.anthropic.com/glasswing

Ben Livengood 1 Apr 2026 21:01 UTC
4 points
0
on: Lesswrong Liberated
Paperclips
Paperclips, and clippy

Ben Livengood 13 Jan 2026 2:36 UTC
5 points
0
in reply to: Steffee’s comment on: Review: Planecrash
On a recent re-read I think I understand a bit better.
It’s true that individual humans can’t realistically avoid giving in to threats or even accidentally threatening others, but institutions can commit to it as a legible position, e.g. “we will not negotiate with terrorists”.
If an irrational entity has the ability to unilaterally destroy the universe then it’s probably going to get destroyed anyway, so it makes more sense to follow through on precommitments in the real world and in counterfactuals to coordinate with actually rational agents.
I think the key is that if we all went MAD legibly at the same time then things would work out a lot better. And refusing to give in to threats doesn’t necessarily mean destruction, it can be as simple as collectively refusing to pay ransomware attackers even though it is currently more expensive, in the expectation that eventually it will be less expensive.

Ben Livengood 23 Oct 2025 21:24 UTC
3 points
0
in reply to: jessicata’s comment on: Homomorphically encrypted consciousness and its implications
After doing some more research I am not sure that it’s always possible to derive a public key knowing only the evaluation key; it seems to depend on the actual FHE scheme.
So the trilemma may be unaffected by this hypothetical. There’s also the question of duplication vs. unification for an observer that has the option to stay at base level reality or enter a homomorphically encrypted computation and whether those should be considered equivalent (enough).

Ben Livengood 23 Oct 2025 20:16 UTC
3 points
0
in reply to: jessicata’s comment on: Homomorphically encrypted consciousness and its implications
To perform homomorphic operations you need the public key, and that also allows one to encrypt any new value and perform further hidden computations under that key. The private key allows decryption of the values.
I suppose you could argue that the homomorphically encrypted mind exists ala mathematical realism even if the public key is destroyed, but it would be something “outside reality” computing future states of the encrypted mind after the public key is no longer available.

Ben Livengood 23 Oct 2025 6:19 UTC
1 point
0
in reply to: jessicata’s comment on: Homomorphically encrypted consciousness and its implications
It’s possible to alter a homomorphic computation in arbitrary ways without knowing the decryption key.
An omniscient observer can homomorphically encrypt a copy of themselves under the same key as the encrypted mind and run a computation of its own copy examining every aspect of the internal mental states of the subject, since they share the same key.
If there are N homomorphically encrypted minds in reality then the omniscient observer will have to create N layers of homomorphic computation in order for the innermost computation to yield the observation of all N minds’ internal states, each passed in turn to a sub-computation, and relying on the premise that homomorphically encrypted minds are conscious for the inner observer to be conscious.
The question is whether encoding all of reality and homomorphically encrypting it necessarily causes a loss of fidelity. If yes, one of the trilemmas still holds. Otherwise there’s no trilemma and the innermost omniscient observer sees all of reality and all internal mental states. I’d argue that for a meaningful omniscient observer to exist it is the case that encoding of reality (into the mind of the observer) must not result in a loss of fidelity. There could be some edge-cases where a polynomial amount of fidelity is lost due to the homomorphic encryption that wouldn’t be lost to the “natural” omniscient observer’s encoding of reality, but I think it stretches the practical definition of omniscience for an observer.
I think the argument extends to physics but the polynomial loss of fidelity is more likely to cause problems in a very homomorphically-encrypted-mind-populated universe.

Ben Livengood 19 Oct 2025 0:39 UTC
1 point
2
on: The IABIED statement is not literally true
If the argument is that 1e9 very smart humans at 100x speed yield safe superintelligent outcomes “soon”, how is that very different from “pause everything now and let N very smart humans figure out safe, aligned superintelligent outcomes over an extended timeframe, on the order of 1e11/N days/years”? It’s just time-shifting safe human work.
I also worry that billions of very smart super-fast humans might decide to try building superintelligence directly, as fast as they can, so that we get doom in months instead of years

Ben Livengood 5 Sep 2025 2:38 UTC
1 point
0
in reply to: RamblinDash’s comment on: If I imagine that I am immune to advertising, what am I probably missing?
I didn’t know Corona had a beach vibe, but I have seen a number of Corona ads. Does this mean advertising doesn’t have much effect on me (beyond name-brand recognition)? I think I associate Corona more with tacos than anything else.

Ben Livengood 6 Aug 2025 22:43 UTC
5 points
0
in reply to: ryan_greenblatt’s comment on: The Problem
Go is in that weird spot that chess was for ~decades[0] where the best humans could beat some of the best engines but it was getting harder, until Rybka, Stockfish and others closed the door and continued far beyond human ability (measured by ELO). AlphaGo is barely a decade old, and it does seem like progress on games has taken a decade or more to become fully superhuman from the first challenges to human world champions.
I think it is the case that when the deep learning approach Stockfish used became superhuman it very quickly became dramatically superhuman within a few years/months despite years of earlier work and slow growth. There seems to be explosive gains in capability at ~years-long intervals.
Similarly, most capability gains in math, essay writing, and writing code have periods of explosive growth and periods of slow growth. So far none of the trends in these three at human level have more than ~5 years of history; earlier systems could provide rudimentary functionality but were significantly constrained by specially designed harnesses or environments they operated within as opposed to the generality of LLMs.
So I think the phrase “do X at all” really applies to the general way that deep learning has allowed ML to do X with significantly fewer or no harnesses. Constraint search and expert systems have been around for decades with slow improvements but deep learning is not a direct offshoot of those approaches and so not quite the same “AI” doing X to compare the progress over time.
[0] https://www.reddit.com/r/chess/comments/xtjstq/the_strongest_engines_over_time/

Ben Livengood 1 Aug 2025 22:20 UTC
8 points
2
in reply to: uugr’s comment on: Do Not Render Your Counterfactuals
It’s the level of detail that’s the real risk. Sora or Veo would generate motion video and audio, bringing even more false life into the counterfactual. People get emotionally attached to characters in movies; imagine trying not to form attachments to interactive videos of your own counterfactual children who call you “Mom” or “Dad”. Your dead friend or relative could emotionally-believably talk to you from beyond the grave.
That’s the kind of thing only the ultra-rich could have conceived of having someone fabricate for them in the past, and it would have come with at least some checks and balances. Now kids in elementary school can necromance their dead parent or whatever.
Realistically, I think it will become “normal” to have your counterfactual worlds easily accessible in this way and the new generations will simply adapt and develop internal safeguards against getting exploited by it, much like we learn how to deal with realistic dreams. I honestly don’t know about the rest of us hitting it later in adulthood.

Ben Livengood 31 Jul 2025 23:45 UTC
5 points
2
on: Follow-up to “My Empathy Is Rarely Kind”
I’m curious what happens if you try on a different suspension of disbelief; imagine peoples’ lives if they lack only growth mindset and not any other moral or agentic abilities.
I find quite a bit of difference in behavior between smart people who believe things about what and who they are, and the people who believe things about how they have acted, can change, and may act in the future.
Smart people without growth mindset often rabbithole into things like legalistic religions and overcoming what they perceive as unalterable weaknesses inherent to their nature, or try to maximize their perceived inherent strengths, ignoring development of new skills and abilities. Introspection is like a checklist of how well they’ve done against a platonic ideal, with maybe some planning to avoid unwinnable situations.
Smart people with growth mindset usually focus on therapy (e.g. understanding their patterns of behavior and the outcomes and how they might alter those patterns in a persistent way), learning new skills and behaviors, possibly some mind-altering substances, and interacting with a lot of diverse other people to understand how they might change or acquire new beliefs and behaviors. Introspection is an exploration of possibilities and personal history and values and how to begin winning in previously unwinnable situations.
Less smart people tend to follow similar patterns, but slower or needing more guidance to proceed.

Ben Livengood 25 Jun 2025 22:19 UTC
1 point
0
in reply to: LGS’s comment on: Consider chilling out in 2028
I think that we have both the bitter lesson that transformers will continue to gain capabilities with scale and also that there are optimizations that will apply to intelligent models generally and orthogonally to computing scale. The latter details seem dangerous to publicize widely in case we happen to be in the world of a hardware overhang allowing AGI or RSI (which I think could be achieved easier/sooner by a “narrower” coding agent and then leading rapidly to AGI) on smaller-than-datacenter clusters of machines today.

Ben Livengood 24 Jun 2025 20:26 UTC
1 point
0
in reply to: LGS’s comment on: Consider chilling out in 2028
There’s quite a difference between a couple frontier labs achieving AGI internally and the whole internet being able to achieve AGI on a llama/deepseek base model, for example.

Ben Livengood 6 Mar 2025 1:02 UTC
4 points
1
in reply to: Julian Bradshaw’s comment on: On the Rationality of Deterring ASI
I think MAIM might only convince people who have p(doom) < 1%.
If we’re at the point that we can convincingly say to each other “this AGI we’re building together can not be used to harm you” we are way closer to p(doom) == 0 than we are right now, IMHO.
Otherwise why would the U.S. or China promising to do AGI research in a MAIMable way be any more convincing than the strategies at alignment that would first be necessary to trust AGI at all? The risk is “anyone gets AGI” until p(doom) is low, and at that point I am unsure if any particular country would choose to forego AGI if it didn’t perfectly align politically because, again, for one random blob of humanness to convince an alien-minded AGI to preserve aspects of the random blob it cares about, it’s likely to encompass 99.9% of what other human blobs care about.
Where that leaves us is that if U.S. and China have very different estimates of p(doom) they are unlikely to cooperate at all in making AGI progress legible to each other. And if they have similar p(doom) they either cooperate strongly to prevent all AGI or cooperate to build the same thing, very roughly.

Ben Livengood 5 Mar 2025 23:30 UTC
3 points
1
on: On the Rationality of Deterring ASI
I have significant misgivings about the comparison with MAD which relies on overwhelming destructive response being available and thus renders a debilitating first-strike being unavailable.
With AGI a first strike seems both likely to succeed and predicted in advance by several folks in several ways (full takeover, pivotal act, singleton outcome) whereas only a few (Von Neumann) argued for a first strike before the USSR obtained nuclear weapons, with no arguments I am aware of after they did.
If an AGI takeover is likely to trigger MAD itself then that is a separate and potentially interesting line of reasoning, but I don’t see the inherent teeth in MAIM. If countries are in a cold war rush to AGI then the most well-funded and covert attempt will achieve AGI first and likely initiate a first strike that circumvents MAD itself through new technological capabilities.

Ben Livengood 4 Mar 2025 23:12 UTC
3 points
0
on: How Much Are LLMs Actually Boosting Real-World Programmer Productivity?
For Golang:
Writing unit test cases is almost automatic (Claude 3.5 and now Claude 3.7). It’s good at the specific test setup necessary and the odd syntax/boilerplate that some testing libraries require. At least 5x speedup.
Autosuggestions (Cursor) are net positive, probably 2x-5x speedup depending on how familiar I am with the codebase.