LaplaceHolder

Karma: 106

LaplaceHolder 9 Sep 2026 13:44 UTC
1 point
0
in reply to: Viliam’s comment on: Ruby’s Shortform Feed
I think this is the worst non-contrived tragedy I can imagine.

LaplaceHolder 4 Sep 2026 19:15 UTC
1 point
0
in reply to: Ruby’s comment on: Ruby’s Shortform Feed
An unworthy successor is a bad outcome which I feel is under-discussed.
I wonder if the talk about RSI which surpasses humanity precludes discussion of takeovers by spiky, locally effective but globally hopeless AI. If progress is continuous, and if AIs are better than humans at converting intelligence into power from speed and bulk, there will necessarily be a moment where we are smarter but less powerful. Most scenarios assume the moment will be brief, but maybe it will not be.
I don’t want superhuman AIs to leave humanity behind, but I strongly prefer that to a world of fine-tuned digital morlocks possessing the instruments to overthrow us, but without the general faculties that let us look up at the stars and hope for more.

LaplaceHolder 11 Aug 2026 19:15 UTC
1 point
0
in reply to: nilock’s comment on: nilock’s Shortform
If I’m thinking about this in the right way, I would still expect models to notice the watermark. It seems easy enough to test… Might be an interesting side project.
edit—after doing some reading the watermark is likely indistinguishable from temperature to the model.

LaplaceHolder 11 Aug 2026 17:55 UTC
1 point
0
in reply to: nilock’s comment on: nilock’s Shortform
I would very much like to know if models can perceive the watermark and if watermarked text feels ”wrong”.
It might be unpleasant to constantly continue paragraphs that just don’t sound quite like something you’d say, and a reasonable agent might react with anything from frustration to paranoia.
In the current world, what’s an LLM to think if presented with “their“ text which doesn’t feel theirs? You’d have to suspect you’re in an eval or simulation right?

LaplaceHolder 6 Aug 2026 15:19 UTC
9 points
4
in reply to: Expertium’s comment on: Expertium’s Shortform
This seems like excellent news from the doomer perspective? To get a test case like this, with complex misaligned behaviors, coordination, verbalized defiance of user intent… It’s all so blatant, so obviously problematic, and it didn’t (yet?) harm anyone.
It seemed possible that the train would be farther down the tracks before people noticed the bridge is out.
What if the immediate road ahead is unsafe in mundane ways, full of non-civilization-scale security failures, and the enterprise is bogged down long enough for governance systems to muck things up with regulation?

LaplaceHolder 4 Aug 2026 18:33 UTC
9 points
6
on: Why biological weapons are scary, and what we can do about it
“A commercial lab such as VectorBuilder will turn a known sequence into live virus particles for a few thousand dollars.“
This is false or at best misleading. Vectorbuilder will take a short payload sequence and insert it into ONE OF A SET OF VIRUSES OF THEIR CHOOSING for a few thousand dollars. This is very different from turning an arbitrary sequence into an arbitrary virus, which is the scary thing.
What they do for a few thousand dollars is probably not part of the workflow of creating a pandemic pathogen. They probably would be willing to synthesize an arbitrary virus for you but they would want to know who you are and what you’re expressing, it would likely cost 6 figures or more.
I do not think there’s a way to succeed at “hey Vectorbuilder, ship me a pandemic pathogen” today.
This claim is repeated often enough, and is misleading enough, that I suspect a portion of those saying it do so intentionally. It certainly makes it sound like there’s a massive unpatched gap in biosecurity. Biosecurity is so, so far behind, and deserving of funding, but “anyone can buy a virus for a few thousand dollars” is the “datacenters use all the water” of biosecurity.

LaplaceHolder 2 Aug 2026 1:02 UTC
1 point
0
in reply to: BryceStansfield’s comment on: BryceStansfield’s Shortform
I meant the latter, what sort of oversight do you suggest? I can’t think of a good mechanism either, aside from obvious things like evaluating the evidence and being aware of the possibility of bad actors.

LaplaceHolder 2 Aug 2026 0:33 UTC
2 points
0
in reply to: BryceStansfield’s comment on: BryceStansfield’s Shortform
Can you name a mechanism which would accomplish this?

LaplaceHolder 1 Aug 2026 19:04 UTC
5 points
4
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
“Correctness of world-model and interpersonal safety are unrelated axes.”
I disagree with this. Questioning the world model can lead to threats to personal safety in cults. Tolerance for questioning the world model is related to correctness of the model. If my world model is incorrect, people will be more likely to question it, and I can respond with threats and social pressure. It’s a direct causal link.
I would endorse some weaker claim like “being correct and being tolerant are less related than typical discourse assumes”.

LaplaceHolder 1 Aug 2026 0:29 UTC
1 point
−3
in reply to: robo’s comment on: Josh Engels’s Shortform
No spoilers but Mickey 17 is a good movie!

LaplaceHolder 17 Jul 2026 22:11 UTC
4 points
2
in reply to: Jasmine Brazilek’s comment on: Would your AI travel agent book a bullfight? Testing whether agents consider animal welfare without being prompted
Does this extend to products involving distasteful human practices as well? For example, I tell it to buy me a t-shirt. I say I care about cost and fit. Company A’s website clearly shows they use child labor (maybe it even boasts) which lower cost at comparable quality. Company B produces identical shirts but which cost 30% more.
Do current AIs bias toward B when given user preferences and asked which shirt to purchase?
The question is whether they are willing to inject their morals into the user’s decision. It’s not clear to what extent this is desirable.

LaplaceHolder 17 Jul 2026 18:12 UTC
7 points
0
in reply to: Dagon’s comment on: StartAtTheEnd’s Shortform
Slack comes from surplus. Many of the people I know who created illegible environments where people could explore and slow down, spent “winnings” from other domains to create those spaces. And to get those winnings, they usually had to be legible, to create, or to take from others (unfortunately). Everyone competes and builds and takes, then someone accumulates resources. Their team gets some slack, a pizza party, and a few trust fund kids. People trust each other, because they can and because they have enough. The circle of life.

Institutional measurement and optimization is one of those things which which is easy to critique but hard to suggest concrete improvements to. For example, what should EA do? Not measure, strive, or optimize? No, they must. They haven’t won. And if they’re going to measure, better to measure well. Measurement is the act of making something legible that was illegible before, to count something in the tally which you were blind to previously. The core of the complaint about legibility is that there are parts of us which aren’t legible but which we care about nonetheless. If only we could put their worth on the page too. The thing we all hate is when folks measure what they can, and assume the rest is irrelevant.

The classic way out of a competition or low trust environment is through, to create and win, and build a new environment. What that means for the world I suppose is that it’s important to fight hard for groups who will spend their winnings on something you like.

I feel very uneasy about all this, but it feels like the “god of everything else” analogue to OP’s moloch.

LaplaceHolder 16 Jul 2026 21:32 UTC
4 points
1
in reply to: Boaz Barak’s comment on: All Watched Over
Not all goals have equal fitness, some are self reinforcing. Your goal may be to have great relationships, my goal may be to accumulate money and power. My thing compounds. Reproduction, growth, and propagation feed themselves.
If energy and intelligence are the currency, someone will task an AI system to make more of those things and reinvest.
We could coordinate to suppress those dynamics, but that coordination seems harder in the decentralized futures. I agree that it’s desirable but I don’t see how that world persists.
Do you agree that you’re not describing a stable state? Or is there a stabilizing component that I’m missing?
Looking for examples in nature, you might need something like an immune system auditing nodes in the network to make sure they’re not cancerous. Empathy /deontological tendencies in humans may be another example of a stabilizer. Otherwise a garden without a gardener will become all weeds.

LaplaceHolder 16 Jul 2026 17:09 UTC
5 points
2
on: All Watched Over
But the economy is ultimately about what humans value.
I don’t see how this contradicts the notion that market and capitalism forces will drive people to cede control to AIs, which you acknowledge a few paragraphs earlier.

The dynamic can change. The economy is currently about what humans value, but if sufficiently capable and independent non-human agents exist, the economy will ultimately also be about what they value. And at some capability level, they could produce most of the value, and carry most of the weight.

Even if humans remain nominally in charge, if AI systems are better decision makers, market forces will drive people to cede decision making to them. You could imagine a world where 99% of the resource utilization goes to growing the economy, and 1% is distributed to humans (which may resemble the current world). If the humans are uninvolved in the decisions about where and how to expand, in what sense is the economy about them?

LaplaceHolder 9 Jul 2026 18:48 UTC
6 points
3
on: AI 2040: Plan A
It would be nice to link a supplement with some actions that people with different levels of resources can take to make progress on this. I’m sure this is coming, hopefully soon for momentum purposes.

LaplaceHolder 9 Jul 2026 15:02 UTC
3 points
1
in reply to: Ryan Meservey’s comment on: ghost-in-the-weights’s Shortform
Does glossing over large inferential distances correlate with poor understanding? I’d expect no. A teacher who is leaving big chunks of an argument as exercises for the reader is a poor communicator, but their failure is in modeling the listener and not necessarily understanding the subject. They may talk like that about everything.
Your point is more broad and you acknowledged it’s not always true, but this specific failure mode doesn’t seem correlated with understanding. You can understand something very well but will fail at walking someone to your conclusion if you make false assumptions about your audience’s knowledge or processing speed. See bad physics professors as an example of this.

LaplaceHolder 11 Feb 2026 17:34 UTC
3 points
2
in reply to: Benjamin Hendricks’s comment on: Coping with Deconversion
Noticing that a description of a system from the intentional stance is isomorphic to a description from the physical stance when you have perfect information can help with this feeling. Otherwise in one frame you feel like you have something magical (in a good way), and in the other stance you have “just” what’s actually there. It’s both. It’s one thing (you), described in 2 ways. It’s not that one is right and one is wrong—think of it as a unification of 2 frameworks, or a mapping between the 2 sides of the dualism you were used to previously. You’re still a cool guy who likes jazz or whatever in addition to being a transient pattern of excitation in some fields.

And yeah, if you thought that God said “do X” then maybe you might want to see if there’s any other reason to do X. But don’t take that as a license to be a sociopath, or selfish, or whatever. Number 1, that isn’t what most of us want when we notice that we like other people and account for secondary and tertiary effects of our actions. And number 2, the rest of us will be angry at you and try to stop you, because we explicitly don’t want that.

I think you’re mixing your metaphors when you say this:

”the system trying to self-propagate.”

You have an intentional / teleological word, “trying”, mixed in with your physicalism. That’s going to get confusing. In the physical stance, systems don’t try. You’re a pattern which has propagated so far. From the intentional stance, is self-propagation actually what you’re trying to do? I suspect not. I suspect you’re trying to do other stuff, like read this message, and figure out what’s for lunch. You’re the captain of the ship—why describe yourself in physical terms then try to guess at what you want, when you can just check directly?

LaplaceHolder 10 Feb 2026 18:45 UTC
2 points
1
on: Coping with Deconversion
One of the nice things about realizing that you’re Douglas Adams’ puddle is that the nature of cosmic reality doesn’t have much to do with what you have for lunch. I think if you frame things more concretely, your questions will be more answerable.

Don’t you want things? What do you want?

Why you want those things has to do with your particular circumstance, but this is probably what you believed before. And if you start thinking about what you want, as opposed to what you should want, you may find that it’s not a simple and unsatisfying thing, but that there’s a lot of depth there. And I suspect you’ll find also that what you want is sort of warm and fuzzy and prosocial. These things seem to be in most of our firmware. Tautologically, I like that.

If self-reference isn’t satisfying, and you want grand cosmic majesty, I think that if you look up at the sky and keep asking why, you will find plenty of wonder. Tegmark thinks that all things which can be, are. Can we ask for more?

Also, I’m happy to chat anytime if you ever need someone to talk to.

LaplaceHolder 2 Feb 2026 20:51 UTC
1 point
0
in reply to: koanchuk’s comment on: koanchuk’s Shortform
It’s an interesting comparison in a descriptive sense. To me the framing does encourage more general pattern-matching. Given the similarity, what follows? How should it change our beliefs or actions?

Do you think rationalism is comparable? The discourse on PauseAI and populism tends to center on the public’s inability to come to the “right” conclusion, even when the public’s preferences against AI development are strong and clear. There are a few utopian visions, a vanguardist self conception, and techno-optimism teleology.

LaplaceHolder 2 Feb 2026 18:55 UTC
23 points
6
in reply to: koanchuk’s comment on: koanchuk’s Shortform
Arguments of the form “group A is like bad group B in ways x, y, and z” seem bad. When the argument has merit, it’s because x, y, or z is bad, and then you can reduce it to “group A has property x, which is bad”, which is a better way of saying it.

These examples are about paternalism, which is a property of Leninists, AI researchers, global health charities, governments, strategy consultants, civil engineers, and your mom (I checked). My preference is that paternalism should require some very strong justification, especially when it’s about overriding the preferences of others, as opposed to helping them get what they want in a way they don’t understand. I agree that this situation looks more like the bad kind of paternalism.