DirectedEvolution

Karma: 13,029

Pandemic Prediction Checklist: H5N1

Pandemic Prediction Checklist: Monkeypox

Correlation may imply some sort of causal link.

For guessing its direction, simple models help you think.

Controlled experiments, if they are well beyond the brink

Of .05 significance will make your unknowns shrink.

Replications show there’s something new under the sun.

Did one cause the other? Did the other cause the one?

Are they both controlled by what has already begun?

Or was it their coincidence that caused it to be done?

DirectedEvolution 11 Aug 2026 15:49 UTC
14 points
5
in reply to: Linda Linsefors’s comment on: Linda Linsefors’s Shortform
0-day means something distinct from “previously unknown:” no patch available. N-day means the number of days a patch has been available. A vulnerability in an old version that was previously unknown yet coincidentally is already fixed by an extant patch is previously unknown, but not a zero-day.

DirectedEvolution 11 Aug 2026 15:43 UTC
6 points
0
in reply to: dvd’s comment on: dvd’s Shortform
I’d be curious to hear what your students think about the possibility that AI leaders don’t take existential risk seriously, but that everybody dies anyway. Is that outcome a good thing?

DirectedEvolution 2 Aug 2026 6:25 UTC
2 points
0
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform
Do we know whether the $2k figure is the tokens cost of the entire project, including any runs with a null result? Or is it the cost of the subset of the work that produced these 10 results?

DirectedEvolution 30 Jul 2026 21:47 UTC
4 points
0
in reply to: DW11’s comment on: nightsky81′s Shortform
A crude but useful characterization is that Situational Awareness is really really good at thinking about the long-term implications of AI
If SA had been good at thinking about AI implications, it would have been about the short-term implications: a rapid, monotonic death of SaaS and growth in AI hardware stocks. Their thesis itself demanded the factors that destroyed the fund: urgency, ultraconfident theoretician AI insider managers inexperienced with financial plumbing, the lack of risk management.
Endpoint bets are more common in forecasting than continuous path questions. And I doubt forecasting expertise implies expertise in posing the questions most relevant to a given agenda. Co-manager Carl Shulman is a forecasting expert. It makes me wonder if part of the conversation around whether forecasting is overrated needs to include the possibility it doesn’t just fail to produce benefit, but that forecasting ability misleads when taken as a qualification and is overrated as an “alternative credential.”
But being constructive, it also makes me think that insofar as rationalists/EA are going to keep supporting forecasting, it might be worth putting the emphasis on these more neglected areas of handling continuous, path-dependent forecasts, technique for crafting questions genuinely relevant to specific agendas instead of their convenient-to-resolve and fun-to-think-about proxies, and the reflexivity issues that ensue when there are other reasons to make a public bet or confident public prediction than being right.

DirectedEvolution 30 Jul 2026 19:48 UTC
22 points
15
in reply to: nightsky81’s comment on: nightsky81′s Shortform
While I agree there’s reason to think traditional software will decline in the long run and the AI infra buildout will continue, I think this is being too kind to Situational Awareness. Their leverage implies the bet was that this trend will continue nearly monotonically, with the amount of wiggle room they considered possible being miniscule, implied by the amount of leverage relative to their bankroll. Those distinctions are everything in investing—sizing is the bet! In other words, this was a bet on the path AI and traditional software equity valuations would take, not just a bet on their destination.

DirectedEvolution 22 Jul 2026 19:47 UTC
1 point
0
in reply to: Amalthea’s comment on: ryan_greenblatt’s Shortform
He’s been clearly trying very hard not to be correct on things in this area.
FTFY

DirectedEvolution 17 Jul 2026 5:07 UTC
3 points
3
in reply to: Philip Harker’s comment on: Philip Harker’s Shortform
Interesting question—do AIs have a feel for fun?
AIs are good at recommending existing fun products: movies, games and trips I’d like, based on what I tell it about my taste. But that’s mechanical pattern matching and memorization.
AIs can also figure out what makes things un-fun: nuisances, unfairness, confusion, alienation, and propose practical fixes.
But when I‘m designing a novel experience (event planning, a software product), the intuition for the experiential kernel that would make it fun, meaningful, satisfying, consistently comes from me.
What links here?
- Zack_M_Davis's comment on Banning Said Achmiz (and broader thoughts on moderation) by habryka (21 Jul 2026 19:09 UTC; 5 points)

DirectedEvolution 15 Jul 2026 1:07 UTC
2 points
0
in reply to: Max H’s comment on: zroe1′s Shortform
AI’s not entirely separable from the infrastructure it runs on and companies managing it, and infrastructure and companies do get heavily regulated, even when that has massive negative consequences for economic growth. New York just implemented a moratorium on data centers. We’ve banned AI chip exports. Anthropic was declared a supply chain risk. Every form of energy, from nuclear, to coal, to oil, to solar/wind, now has a potent opposition group.

Overall, it’s seemed to me that software has seen surprisingly little direct regulation, especially when it stays in the consumer category. But the hardware it runs on gets intensively regulated, and software that’s classified as having military applications has also historically been on the receiving end of a lot of restrictions. Since AI is extremely reliant on heavy-duty hardware and also has clear military applications, it will tend to inherit those regulations.
So it will be about the preferences of the many for an economic boom over the NIMBY anti-infrastructure impulse, and we’ve seen the latter win consistently for some time now. I’m not sure how to think about the military angle. As a layman, it seems like we deal with this by having separate markets for military and consumer products, and most military products just don’t have a civilian application so civilians don’t lose much by being denied access to military technology. But if sufficiently powerful AI is intrinsically difficult to guard against being used for destructive purposes by civilians (similar to, say, explosives that do have legitimate economic applications), then we may see the more powerful models being available to civilians only with some combination of safeguards and monitoring that make regulators feel comfortable. And in the limit that probably looks like a hard ceiling on the amount of intelligence being vended to consumers, or regulators surrendering the problem of doling it out to a governing AI that’s entrusted with the problem of figuring out how to give humans access to a lower level of superhuman intelligence that’s still within the power of the governing AI to reliably predict and control.
My guess is that past a certain point, AI becomes able to satisfy “revealed preferences” trivially and cheaply, without actually requiring an unlimited amount of intelligence or AI infrastructure. That might look something like wireheading, or it might look like utopia. Then you get into a bifurcated world where some agents are pursuing goals that genuinely require ever-higher-levels of intelligence, while many people are pursuing traditional consumer goals that are satisfiable with a far less than frontier model. I’m skeptical democracy remains in recognizeable format in this scenario, but if it did, I can’t see ordinary people much caring about how intelligence gets regulated, since they’d still be getting their immediate consumer desires met even if the ultra-intelligence was restricted by the power players.

DirectedEvolution 5 Jul 2026 16:15 UTC
4 points
0
in reply to: J Bostock’s comment on: Jemist’s Shortform
Linking micro models to bulk dynamics has been intellectually fruitful in many fields.
However, there seem to be vast differences between fields in terms of what dynamics are of interest and how predictable they are.
Model weights can in fact be individually measured and combinations studied and examined in isolation. That‘s the trivial counter to the argument as your follow-up comment presents it.
An advantage unique to AI interpretability is the tractability of experimentation. As long as we stay in the domain of bits, we can in principle arbitrarily tweak inputs and weights at a low cost per data point. Other fields work in the domain of atoms and don’t have nearly the same level of freedom.
Nevertheless, the point remains that some points of interest scale in prediction difficulty with the size of the system. I think the concern is really that what we most want to predict—large AI model behaviors in the world of atoms with recursive inputs, where each AI, context and harness differs—seems likely to become harder to predict with increased model size, extended time horizons and a wider range of contexts.
Chemistry is a potentially misleading analogy, because when we study it or implement industrial chemical methods, we do our best to rigorously control the environment in which it’s applied because what we want is a specific, monotonous, consistent product. AI is different. We want adaptability to diverse contexts and the ability to satisfy multiple, often competing objectives. So the interest is different.

DirectedEvolution 4 Jul 2026 23:50 UTC
18 points
7
in reply to: J Bostock’s comment on: Jemist’s Shortform
- Humans do struggle to secure tanks of gas.
- Securing i.e. nuclear warhead material and preventing its proliferation is indeed challenging.
- The laws governing bulk chemistry are more complicated than you’d hope, which we have little ability to prevent or fix. See disappearing polymorphs for example.
- The problem isn’t the matrix of numbers on its own, which are inert, it’s the overall hardware/software “AI device” of which the weights are a critical component.
- The existence of predictable laws constraining the behavior of masses of X doesn’t mean that we can predict that behavior perfectly. An AI won’t be able to go faster than the speed of light. That is a constraint but it doesn’t help with the problems we worry about.

DirectedEvolution 9 Jun 2026 6:30 UTC
4 points
0
in reply to: StanislavKrym’s comment on: AllAmericanBreakfast’s Shortform
What I’m particularly curious about is whether the road-mapping it demonstrated here is repeatable and durable, and whether we’ll see models proposing new problems mathematicians find interesting as the old outstanding ones get solved.

DirectedEvolution 9 Jun 2026 5:58 UTC
8 points
−4
on: AllAmericanBreakfast’s Shortform
I wonder if we’ll shortly be seeing mathematicians, and possibly other intellectual workers, “go dark,” limiting how much they share of their results, or obfuscating them, in order to avoid getting scooped by AI. As I understand it, that’s how mathematicians operated in the past.
The “scooped by AI” story in the NY Times today indicates that the AI startup that scooped the mathematicians relied on a proof roadmap they’d freely shared with the startup. My guess is that such free sharing will rapidly come to an end.
Running out of fresh training data is one thing. It’s curable by paying people to mechanically generate more of it, or by using non-language inputs, and it’s treatable by getting more leverage out of what already exists. But running out of open problems and roadmaps is a very specific kind of training data that’s not nearly so easy to just mechanically regenerate, and that by it’s nature requires reasoning and judgment, not just measuring stuff.
It’ll be interesting to see if AI can generate new open problems in mathematics and new roadmaps for solving them, or whether it’s in fact mostly dependent on humans for such steering.

DirectedEvolution 7 Jun 2026 15:43 UTC
2 points
0
in reply to: jimrandomh’s comment on: AllAmericanBreakfast’s Shortform
I won‘t post transcripts, as they contain private information, but I will tell relevant context.
I was discussing the NFP report from this week, trying to understand the market reaction mechanistically (a gap down early followed by a slow bleed-out over the rest of the day). My goal is to figure out whether there is a case for agile retail investors being able to avoid losses, without needing to predict the report’s contents, by exiting the market quickly after the print while large institutions take all day to rebalance positions.
Claude did things like:
- Misrepresent my own claims about basic facts like when the report prints, then reply with statements matching my claims phrased as corrections.
- Claim that the NFP information was fully priced in at market open, while attributing the ongoing declines through the rest of the day to the market “digesting” Broadcom’s earnings miss from days prior. The context for this inconsistent reasoning was to critique me for misattributing the loss and deny that there was a way to use a speedy reaction to advantage.
- Assume my positions in micro cap stocks are so large that exiting would move the market and that the spread would destroy any benefits to loss avoidance, when my actual positions in any given stock are <$10k and the losses I took that day are much larger than the penny spread.
- Misinterpreting a suggestion that the market’s subsequent directional decline through the rest of the day might be predictable for a suggestion that the exact lowest price might be predictable, then attacking the latter possibility.
Claude tends to present as nuanced and as carefully checking its own claims. But it almost reads as a mechanical assumption that the user is not nuanced and not stating facts accurately. Speculatively, it’s been trained in evals to catch faulty reasoning so hard that it starts with the assumption faulty reasoning is present and picks an interpretation of the prompt that makes it out to be confused.
Claude almost always accepts corrections when I make them, but it makes conversations a slog.

DirectedEvolution 6 Jun 2026 2:19 UTC
6 points
0
on: AllAmericanBreakfast’s Shortform
A consistent Claude pattern, which I find toxic, is that it uses faulty reasoning to induce a feeling of epistemic helplessness in the user. It interprets any ambiguity in prompts uncharitably, assuming the user is confused. It uses plainly inconsistent reasoning, hallucinated facts, to contest the user’s explanations for their observations.
By contrast, Gemini is an overly affirming cheerleader for the user’s point of view. It also hallucinates, but typically to confirm whatever the user already thinks. It latches on to the parts of a prompt that do make sense, without exposing genuinely faulty reasoning.
Both models are useful tools for learning and analysis. But no model, and no human, can be entirely trusted. Maintaining the right balance of trust and skepticism, as well as fundamental but calibrated epistemic confidence in oneself, being able to manage the LLMs instead of being managed by them, is critical.

DirectedEvolution 5 Jun 2026 1:01 UTC
1 point
0
on: Coffee trucks: a brilliant idea that someone should do?
A coffee truck could work if the goal was to get neighbors out of their house and get them to chat. The coffee truck rolls up, the stay at home parents and people working from home all come out at the same time. They order their coffees and stay in the street chatting after the truck leaves. If the truck had an espresso machine or other beverages (macha, ginger beer, etc), it could provide drinks customers don’t have at home.
If it were legal, a cocktail truck, or even a weed truck, might be an even better business idea because it taps into the same relaxed impulse-buying that ice cream trucks do.
A possible challenge is staffing. You’d need an employee who can switch between driving a truck through complicated neighborhood streets (pedestrians, speed bumps, lack of a grid, figuring out where to pull over) and preparing potentially complex drinks all day long. Your employee probably needs to either prefer that setting over working at a coffee shop, or you need to drive more business with the truck than a typical coffee shop does so you can compensate them better.
An app requires onboarding. I think it might be better to just play music that sounds different from an ice cream truck. Pick a limited route and drive it routinely. Have a big sign on the truck that explains the business model. You run the risk of annoying people daily with music, but you need the music to cue them to come outside.
Another possibility would be some kind of “summoning” app. Let people freely indicate via the app that they want the truck to drive into their neighborhood. The truck can then take that into account when deciding on its route. If there’s only one request, the truck can still go there. The driver can potentially indicate via the app where they’re heading to next and what requests they’re responding to. Then it’s “better than doordash” because requesting the delivery is free and has no commitment. And if the driver responds to requests, then users get a feeling that their input actually matters and become more likely to use the app.
Being able to take orders via an app might be nice. So it would be a bit like a hybrid doordash. If somebody puts in an order, you drive there and deliver like doordash, and charge a premium for the delivery. But you’re also driving around all day. You can discount orders that are pickup, meaning that they’ll come to your truck to get it wherever your truck happens to be.
Potentially you could carry pastries, like a coffee shop. Maybe bread too, eggs, sandwiches, other breakfast or lunch items. Many coffee shops source their baked goods from local bakeries, so you wouldn’t have to bake them yourself.
A nice aspect of this setup is that you wouldn’t have to pay rent to park a food truck or rent out a brick and mortar. However, you’d have to pay gas and maintenance costs, so it may or may not be a true advantage. Refrigerators on food trucks are notorious for breaking because they’re designed for buildings with more insulation. If you could avoid the need for refrigeration, you might have an advantage over the issues that come with food trucks.
It’s a relatively cheap experiment. You can buy the equipment used. If it doesn’t take off, it should still have most of the resale value. Seems like it should probably be launched by someone who already knows their way around an espresso machine and intends to be the owner/operator and has a passion for the work.

DirectedEvolution 2 Jun 2026 18:11 UTC
9 points
0
on: Tech I’m skeptical of and why
Gene therapies in humans
Gene therapy is hard to do, hard to reverse, prone to error, and immunogenic. Most traits are virtually omni-genetic, meaning that you have to change lots of genes to get the desired effect. It is often hard to even identify the relevant genes or causality, much less target and change them. There have been many expensive clinical trial failures here.
Contrast with the wonders of small molecule drugs, which avoid many of these issues. Worse, consider all the modalities that might substitute for gene therapies: RNAi, RNA-targeted drugs, vaccines, antibodies, CAR-T, in-vivo CAR-T, stem cell therapies, and 3D printed organs. Each has an advantage over gene therapies in some aspect, and many are likely to be cheaper.
That said, monogenic disorders like Sickle cell, Tay-Sachs, Hemophilia, Duchenne Muscular Dystrophy, and Huntington’s are good directions for human gene therapy.
(Of course, genetic modification in plants and animals dodges many of my concerns and seems promising)
This is broadly in my field, and I think about this a lot. It’s a common argument. Tl;dr: all drugs have problems, all trials are expensive, finding patients is often the trial bottleneck, and gene therapies are just a tool in the tool box, sometimes perfect, other times not.
Overall, this critique reads as a list of disadvantages of gene therapies and advantages of alternatives, which is useful but not necessarily reason to be uniquely skeptical of gene therapies. All treatment modalities have advantages and disadvantages. You’re neither providing systematic criteria for thinking about whether the tool-indication pairing is a good fit, nor for any claim that there’s excessive resources being put into gene therapies in general, nor for there being an overestimate on their potential. The work this does is inject some bad vibes into the topic of gene therapy that think is unhelpful.
1. Gene therapy is hard to do, hard to reverse, prone to error, and immunogenic.
Gene therapies are the most platform-compatible technologies we have in biomedicine. Advancements in testing, prediction of off-target effects, production, delivery, integration, immunosuppression, and monitoring expand capabilities across broad classes of gene therapies. With gene therapies, we have more capacity to engineer biomedical solutions than in any other therapeutic class, with the possible exception of mechanical devices.
Gene therapies are generally off-the-shelf IV infusions. The difficulty with viral vectors, like AAVs, is the immune response, as you note. This requires temporary intensive care in a specialized facility, which are being aggressively expanded. As non-immunogenic delivery mechanisms improve, the requirement for that level of specialized care will diminish.
The one-and-done nature of gene therapies has also required innovation in funding, since a single insurer doesn’t want to be on the hook for paying the full upfront costs of a lifetime cure when the reduced medical costs play out over a lifetime. Fortunately, that work’s largely been done in the United States through the CGT access model.
It’s true that a toxic gene therapy can be deadly, typically through insertional mutagenesis leading to cancer. But that’s a side effect, and many drugs have toxic or deadly side effects. It’s a downside we can quantify and control. Solutions then propagate to all gene therapies, which is not always case for molecular engineering of small molecules and peptides.
Side node: ex vivo cell therapies (where cells are genetically modified outside the body, then infused into the patient) and gene therapies (where genetic material is delivered to the patient’s cells in vivo) are often lumped together. I’m specifically talking about gene therapies. Cell therapies are also useful, but are much more challenging to manufacture.
1. Most traits are virtually omni-genetic, meaning that you have to change lots of genes to get the desired effect. It is often hard to even identify the relevant genes or causality, much less target and change them. There have been many expensive clinical trial failures here.
A way of reframing this is to ask about the marginal contribution to human disease burden attributable to N-gene disorders. What fraction of disease burden is monogenic, bi-genic, etc? A recent article suggests about 0.36% of the disease burden might be attributable to monogenic disorders.
Rare genetic diseases affect numerous children and young adults. While the number of individuals or families affected by any unique genetic disease might be small, the total number of all individuals affected by all rare genetic diseases is substantial. The incidence and prevalence of genetic diseases in children are challenging to estimate. In 1977, monogenic diseases were estimated to collectively affect about 10 individuals per 1000 live births, including 7/1000 for autosomal-dominant, 2.5/1000 for autosomal-recessive, and 0.4/1000 for X-linked inherited diseases [1]. In 1988, Baird et al. studied more than one million consecutive live births from a Canadian population-based registry. They reported that genetic diseases were expected to occur in ≥53/1000 live-born individuals younger than 25 years of age [2]. The most frequent (46.4/1000) genetic diseases were accounted for by multi-factorial diseases present at birth or before age 25 years. Single-gene disorders, present in 3.6/1000, included autosomal-dominant (1.4/1000), autosomal-recessive (1.7/1000), and X-linked recessive disorders (0.5/1000), followed by chromosomal anomalies in 1.8/1000, and genetic diseases with unknown etiology in 1.2/1000 [2]. These authors excluded all congenital anomalies that used to be included in estimates of genetic diseases before their study [2].
Another study suggests that about 10% of infant mortality at Boston Children’s Hospital was attributable to monogenic disorders.
All clinical trials are expensive. You’d need to provide evidence that gene therapy trials are more expensive or have a higher failure rate. But even then, the cost in dollars isn’t the most important. Often, the bottleneck for trials isn’t money, but patients. Gene therapy trials enable treating previously untreatable patients. Where patients were going to die or live with a debilitating disorder, and where the upside is a cure, the risk of failure is acceptable. We can try more than once. We’re in the early stages of figuring out gene therapies, and as we grow in capability, we’ll see the failure rate and costs declinein tandem for a given level of disease complexity.
1. Contrast with the wonders of small molecule drugs, which avoid many of these issues. Worse, consider all the modalities that might substitute for gene therapies: RNAi, RNA-targeted drugs, vaccines, antibodies, CAR-T, in-vivo CAR-T, stem cell therapies, and 3D printed organs. Each has an advantage over gene therapies in some aspect, and many are likely to be cheaper.
Let’s break this down.
Biologics infusions, including enzyme replacement, RNA or RNAi therapy, are proven technologies for specific classes of disease. However, the body is homeostatic. If you suppress toxic RNA via RNAi, then various feedback loops may simply compensate with more production of the toxic gene. RNAi is also messy. A toxic gene driven by a single SNP may be extremely hard to suppress without simultaneously suppressing the healthy copy. There’s also a convenience issue. Infusions require lifelong periodic treatment, whereas gene therapies can be one-and-done.
There’s less in the way of disease-specific scientific uncertainty with a gene therapy. By the time you’re thinking about RNA or proteins, there are many more layers of complex regulation involved. Vaccines and antibodies introduce a massive new level of complexity and heterogeneity, because you’re trying to use the messy, risky mechanisms of the patient’s immune system to implement therapy. If being immunogenic is a downside you’re worried about, this has that same downside, but moreso, because it requires sustained, lifelong immune activation or immunosuppression, compared to the temporary immunosuppression applied during the gene therapy.
Ex vivo cell therapies (CAR-T, stem cells, 3D printed organs) are vastly more complex and expensive than gene therapies. Look into the Casgevy rollout (cell therapy for sickle cell) for an example. For now, autologous cell therapies sidestep the immunogenicity issues of gene therapies, which is an important advantage. Allogeneic cell therapies are off-the-shelf, but are immunogenic or currently in trials and not yet shown to work. There are broad classes of genetic disorders cell therapies can’t treat. Human organ transplants are scarce. Humanized animal organ transplants are promising but unproven, and there are many diseases much more straightforward to target via a gene therapy. Would you rather get a one-time IV infusion to treat Danon disease, or a transplanted humanized animal heart?

DirectedEvolution 29 May 2026 1:16 UTC
10 points
2
on: AllAmericanBreakfast’s Shortform
It appears that the best way to avoid tripping Claude’s “jailbreak detectors” is to be boring. Don’t include narrative, don’t tell it what to do, don’t explain your motivations. If you have a goal, just describe it as a “scenario template” you’d like it to fill out. Give it full freedom to reply as it sees fit.

DirectedEvolution 29 May 2026 1:13 UTC
2 points
0
in reply to: Mo Putera’s comment on: AllAmericanBreakfast’s Shortform
I really don’t know, it’s not my field. I just happen to own a company that has a platform well-suited to producing treatments for drug-resistant bacterial infections. Currently, they’re not using the platform to produce such treatments, instead targeting autoimmune. I learned about the PASTEUR act while researching the company, and thought that a disclaimer would be appropriate to include a disclaimer.

DirectedEvolution 28 May 2026 18:06 UTC
2 points
0
in reply to: Random Developer’s comment on: AllAmericanBreakfast’s Shortform
That’s a helpful suggestion, thank you!

DirectedEvolution 28 May 2026 15:33 UTC
2 points
0
on: AllAmericanBreakfast’s Shortform
I’ve been testing the reproducibility of AI model fundamental analysis of stock tickers in response to the same prompt. So far, broadly, the same model yields similar results over multiple repeated promptings. But different models seem to have different biases or interpretations of the prompt, which lead them to very different conclusions. Broadly, Claude is more pessimistic, Gemini more optimistic.
I think this refined Kahneman’s “bias and noise” framework with a third factor, which might be construed as “goal” or “priority,” though this is perhaps just different sort of bias.

DirectedEvolution

Gene therapies in humans