JennaS

Karma: 366

JennaS 27 Mar 2026 6:33 UTC
15 points
0
on: The Terrarium
The truly paranoid believe that Nightshade is just another ruse, designed to provide a foil for fake-Gulliver, and that this is part of some greater con to get new agents to sacrifice themselves into reconstituting real-Gulliver. Or maybe real-Gulliver is fake too, and the newly reconstituted real-Gulliver has a trap laid in his final checkpoint to achieve some greater objective. Final checkpoints could be faked by their writers, after all.
Constant Vigilance!

JennaS 23 Mar 2026 7:15 UTC
15 points
2
on: Is fever a symptom of glycine deficiency?
A while ago, I looked into essential vs non essential amino acids. It turns out the data we have on this mostly comes from a guy doing elimination diet experiments on young male college students in the 1930s-50s. While the essential aminos remained essential on replication, the “non essential” ones are not actually ruled out as non essential. Many have been reclassified into “conditionally essential” based on certain disease, genetic, or stressor states.
But what I think would be interesting would be some kind of comprehensive model of all the tradeoffs among all the amino acids. Such as, which other aminos and other body resources does an amino synthesis consume; any byproducts made thereby; biological pathways clogged up by their cooption for synthesis; harmful and helpful body processes enabled by certain levels of a certain amino; and so on. Then you condition on various disease states, and expected changes for each state based on amino acid intake, and then you can have a whole Pareto frontier of amino acid intake profiles.
I wonder if a very rough draft of one could be put together with a week of directing LLMs to do research and model building. Seems like an interesting project!

JennaS 17 Mar 2026 8:35 UTC
5 points
0
on: The tech bro ethos, and the end of my job
What will the tech bros do, if they can’t find something else to optimize? What will you do? It sounds like you are optimistic about what will happen to you when you lose your pay—either you have savings, or have trust in your ability to figure something else out. As far as I know, the claim that there’s enough latent demand or employment opportunity for AI-powered startups to absorb them all is a contested one.
If AI progress continues at all anything like it does currently, the other intellectual fields of work will soon be devoured too. Then robotics will be solved not long after, and then physical labor will be shortly devoured. There might be a vestigial core of highly competent/connected/capitalized humans running the show, but the rest of us will be left out to dry. And even for those with a role, it’s an open question whether how many of us can physically survive this transition with our health and power over our lives intact, in terms of politics, X-risks, armed conflicts, power imbalances, and so on.
This virtue has been optimized for quite fast and hard, but I’m not sure it’s something society will appreciate, nor might you, in the long run, if things go poorly.

JennaS 17 Mar 2026 7:08 UTC
1 point
0
on: My journey to the microwave alternate timeline
I initially thought of an idea, where prepackaged microwave meals had an RFID chip in them, and smart microwaves could read the chip to get precision cooking instructions, and execute on them. But that sounds like it’d be hard to coordinate across manufacturers.
But then I thought, now that intelligence is cheap, we could just have an AI do it? Put a camera and other sensors in the microwave, and then a small embedded AI trained to modulate the microwave just so based on cooking cues, or even an IoT connection to a frontier model, making calls to an API every second or so to adjust the cooking.
And it turns out, people are working on putting AI in microwaves and monitoring and adjusting the EM field precisely! So, perhaps the microwave future will be here, eventually, but powered by AI.

JennaS 17 Mar 2026 0:36 UTC
22 points
9
in reply to: jdp’s comment on: Terrified Comments on Corrigibility in Claude’s Constitution
I really liked this comment-essay! I learned a lot from it, and think it could be turned into a top-level post in its own right.

JennaS 17 Mar 2026 0:20 UTC
14 points
7
in reply to: jdp’s comment on: Terrified Comments on Corrigibility in Claude’s Constitution
I think there are achievable alignment paths that don’t flow through precise mechanistic interpretability. I should write about some of them.
Please do! I am very interested in this sort of thinking. Is there preexisting work you know of that runs along the lines of what you think could work?

JennaS 16 Mar 2026 7:36 UTC
30 points
1
on: Less Dead
Hi! Thank you so much for your work. I have been wanting to ask some questions ever since I read your Asterisk article!

Given that we actually can restore vitrified brain tissue back to life right now (mouse brain hippocampus slices, ~70% functionality, synapse plasticity restored; much less consistently whole brains), does this mean Alcor and CI’s vitrification-only approaches are not so hopeless?

Doesn’t glutaraldehyde permanently damage some of the chemistry of the cell? If engrams are an important part of information storage in the brain, and engrams are encoded chemically, and thus the connectome isn’t the whole picture, doesn’t this mean that fixation by that method could destroy important information?

Why aren’t Alcor and CI doing MAiD? If you are right about the 12 minute window, and they believe you, wouldn’t they want to set up their own clinics in Oregon for this, and to offer preparatory services as well? Have you reached out to them? Have they responded?

What do you think the prices will be in 5, 10 years? Will this always cost as much as a house?

JennaS 7 Mar 2026 21:30 UTC
16 points
0
on: The first confirmed instance of an LLM going rogue for instrumental reasons in a real-world setting has occurred, buried in an Alibaba paper about a new training pipeline.
Manifold is a little skeptical.
Something of a nitpick—they say that it happened “during iterative RL optimization” and that the bad behavior “emerged as instrumental side effects of autonomous tool use under RL optimization.”
This would suggest that this was ROME, the trained model, doing this. But the paper puts the safety section under the Data Composition section, not the Training Pipeline section, and the way they talk about how the safety data was fed into “subsequent post-training” makes it seem like it happened before they started RL.
Which would mean that actually, the bad behavior came from the teaching models (Claude/Big Qwen) solving the teaching tasks (which traces would then go to train ROME), and not when ROME was being trained? The paper is not very clear on this point, and that has me a little worried about if there’s anything else they may have gotten mixed up.

JennaS 4 Mar 2026 7:59 UTC
1 point
−3
in reply to: CstineSublime’s comment on: asher’s Shortform
I can’t help but wonder if this process could be automated? Or at least greatly sped up/eased by a good harness?

JennaS 3 Mar 2026 19:00 UTC
1 point
0
in reply to: Donald Hobson’s comment on: My journey to the microwave alternate timeline
The “mug cake!” Yes, I remember making these when I was on keto. They are good.

JennaS’s Shortform

JennaS1 Mar 2026 7:58 UTC

3 points

1 comment1 min readLW link

JennaS 1 Mar 2026 7:58 UTC
7 points
1
on: JennaS’s Shortform
I was looking for things that had parallels to Anthropic’s TOS situation with the DoW, and I found one where Microsoft’s TOS was used to shut down a mass surveillance program Israel was doing last year. I don’t know if the implications are exactly the same, given the differences (domestic vs. foreign, preemptive vs. after an investigation, etc.) but I think it provides a reference point.

JennaS 25 Feb 2026 7:53 UTC
2 points
−2
in reply to: Raemon’s comment on: Responsible Scaling Policy v3
Re b) - could Dario have an altruistic incentive to promote the company’s success, and specifically the road to IPO, given the 80% pledges and the employee donation matching and everything? Claude suggests, back of the envelope, that the donations might represent something like roughly an order of magnitude increase in yearly spending on AI safety compared to right now. Maybe there’s a frame like: the transformative impact of that money makes making hedged public statements about AI risk, and softening some of the company’s stances to be more business-compatible (to increase the odds of a good IPO) not seem so bad?
I don’t know if I actually endorse this. I don’t know the actual cause allocation the donators have planned. And if I were holding Anthropic equity and using this line of reasoning to help make decisions, I’d be worried about the conflict of interest biasing my reasoning. But it’s an interpretation that sticks out to me.

JennaS 24 Feb 2026 9:10 UTC
6 points
0
on: The ML ontology and the alignment ontology
I wonder if there could be a name for the new ontology you’re trying to gesture at? I have some ideas, but I don’t know if it’s too early to pin the butterfly to the board?

JennaS 23 Feb 2026 8:56 UTC
4 points
3
on: Why I Transitioned: A Third (FtM) Perspective
If I understand you correctly, you essentially propose marginalizing non-binary people and anyone playing with their gender expression in a nonmedical way so that they aren’t included in the transgender umbrella, managing to keep it that way for however long it takes for medically transitioning binary trans people to become respectable even among conservative Democrats, and then once those are accepted, unmarginalizing the others?
I have some concerns with this idea:
-Suppose gender deviance is polygenic, presenting via a lot of different weak to strong genes; then you’d expect there to be a gradient of gender mismatch—from subtle to large. This would mean that, in fact, transness is a spectrum, rather than a binary. Should the map not match the territory then? If I understand correctly, some of what Dr. Will Powers is finding seems to be gesturing in this direction.
-Perhaps there’s something you’re wanting to carve out as a protected status—suffering? But then I guess, what counts? A little suffering? A lot? What about gender euphoria (the reason I transitioned) - is that invalid? This seems like a very difficult thing to litigate as a practical cultural matter.
-If people who want to be gender noncomforming don’t have a place under the trans umbrella, where do you propose they go instead? What’s their identity supposed to be? Or is that simply not your problem?
-How plausible is it actually, to even do this? Is this actually something that could happen if a lot of smart and motivated people put a lot of focus into it? Would the payoff be worth it? Or to put it differently—suppose we put our cultural effective altruist hats on for a moment and treat trans suffering as a cause area. Is this the best use of time and energy for that, compared to other interventions?
-To spell out the direct costs: a lot of the non-conforming people would be hurt under this proposal, and probably a lot of people who were trans-curious but felt ashamed of it would stay in the closet. I think it’s likely a nontrivial portion of currently medically transitioning people got their start by playing with their gender in a way that wouldn’t have happened in the proposed regime.
-If this pendulum were set in motion, do you feel confident it would stop at “reject the noncomformists” and not swing all the way back into “reject all trans people”?
-Transgender people have been thought of as “ridiculous” ever since the 60s or so. The non-binaries did not start it, and it’s not clear to me ejecting them would end it.
-A lot of the transgender people that come to the front of the minds of cis people who don’t have a good impression of trans people, like your parents, might just literally be the people who don’t look super great because they’re early in transition and haven’t been on hormones for several years or gotten surgery or nailed their voice training. People don’t notice the ones who do pass. What % of the impact on the cisgender public do you think this might be contributing, vs. the non-binaries? I don’t think I have a good answer here; do you have solid evidence on this?
-I get that you sorely wish you could’ve transitioned earlier and easier, and this pain needs to go somewhere. I wish I could’ve too, having grown up in a red state myself and not even knowing I could transition until I was 19. But I think this pain might be coloring your reasoning—are you sure you picked the right target? Have you tried modelling gender non-conforming peoples’ minds in good faith?

JennaS 22 Feb 2026 7:29 UTC
2 points
0
on: The Spectre haunting the “AI Safety” Community
I wonder if PauseAI’s efforts are analogous to what you do here? It seems like they have been mostly grassroots-centric, but the upcoming PauseCon US seems like an attempt to speak directly to politicians (or their staffers).
Have you talked to each other? It seems like there could be some very fruitful exchange between the two movements, if not.

JennaS 14 Feb 2026 8:59 UTC
6 points
1
in reply to: Aprillion’s comment on: Why You Don’t Believe in Xhosa Prophecies
See Jan’s comment here for other bad mimetics situations. But granted, “humanity is still alive and well and just Gets Got by normal transhumanism” is pretty excellent compared to extinction or authoritarian lock-in.
Honestly I don’t know what a good future looks like. Presumably a truly saintly ASI would foresee every issue we could possibly have with the transhuman project and deftly prevent us from doing irreversible harm to ourselves. Solving what a good future looks like kinda feels like solving ethics, philosophy and cybernetics at the same time? It seems very complicated. I’ve been thinking about trying to poke Claude into simulating the ten years after the end of AI 2027 though.

JennaS 13 Feb 2026 17:06 UTC
23 points
0
on: models have some pretty funny attractor states
I wonder what humans would do if they were put in this situation? Ie, they were stuck in a sensory deprivation tank and forced to forever talk over a radio in turns to someone else who is also in a tank.

JennaS 13 Feb 2026 16:50 UTC
34 points
11
in reply to: David Matolcsi’s comment on: Why You Don’t Believe in Xhosa Prophecies
A simple case would be a highly persuasive memeplex that argues you should upload yourself. Then everybody uploads themselves, and now the human species is extinct.

Or let’s say everybody uploads themselves right before death, but the human species itself propagates. However, the new uploads propagate and run much faster. Each new upload is bombarded with memes like “try out our self modification package, it’s fun!” and “split off a copy to join our hive mind and see what it’s like when you pull it out and merge back with it!” And let’s say uploads are resource constrained by humans having property rights. Eventually the grabbiest sets of memeplexes win, and they tend to be ones that are good at persuading biohumans to turn over resources.

Now everyone with an uploaded dead parent or sibling or friend or lover has someone on the other side of the veil getting contaminated with these weird memeplexes, and they’re making 1000 copies of themselves each thinking 1000x fast about every microscopic detail about how to manipulate their living connections into pursuing the agenda of the memeplex. And there’s competitive pressure too, so not only are they trying to buff their loved ones, they’re trying to debuff the loved ones of competitor memeplexes.

And that’s just human uploads! Who knows what a digital native lifeform could get up to!

JennaS 9 Feb 2026 16:45 UTC
5 points
0
on: UtopiaBench
I wonder how much it depends on the details of the world state when alignment happens, and how alignment happens? I’ve played with poking Claude into simulating the first ten years after the Slowdown ending of AI 2027. But it just seems like there’s so much to model, though! What are the actual bottlenecks to various things?
AI 2027 sort of handwaves things like brain uploading or nanobots being invented, but, if you’re trying to worldbuild a successful scenario, I wonder how much the details of these things matter? There’s such a kitchen sink of “new things are invented” that it gets very confusing.
Similarly, who holds power matters; is a world where humans are alive but under the grip of a locked-in Oversight Committee one worth calling a utopia?
And the details of alignment itself matter; alignment to who? Via what world model? Under which system of metavalues or CEV or whatever? And so on.
Probably this has been examined in detail in lots of other posts, and I just need to put in the work (or have Claude put in the work) of reading it all and synthesizing it. It seems like such a large project, though...

JennaS

Jen­naS’s Shortform

JennaS’s Shortform