DirectedEvolution

Karma: 12,520

Pandemic Prediction Checklist: H5N1

Pandemic Prediction Checklist: Monkeypox

Correlation may imply some sort of causal link.

For guessing its direction, simple models help you think.

Controlled experiments, if they are well beyond the brink

Of .05 significance will make your unknowns shrink.

Replications show there’s something new under the sun.

Did one cause the other? Did the other cause the one?

Are they both controlled by what has already begun?

Or was it their coincidence that caused it to be done?

DirectedEvolution 17 Apr 2026 22:26 UTC
18 points
3
on: AllAmericanBreakfast’s Shortform
Bad incentives are preventing Americans from using air purifiers and far-UVC to reduce sickness in schools and daycares.
- Schools often aren’t penalized linearly with the number of absences. Short-term occasional absences from illness may have no impact on their budgets. Improved student scholastic performance due to reduced illness may take years to manifest and actually decrease the amount of funding they receive. Any effect of reduced illness on improved scholastic performance will be obscured by many other factors on a per-school basis, so that benefits can only be quantified through scientific efforts.
- Daycares are often small, corner-cutting businesses and are already in high demand and not that professional or organized. Investing in sanitary measures might improve the bottom line, but it takes effort and time to realize any benefits and the costs are paid upfront.
- Trying to push costs onto families results in a freerider problem, where individual families are incentivized to duck the costs of paying for sanitation in public spaces while benefitting from it personally.
- States often gate access to grant money for air quality improvements behind time-consuming applications requiring up-front expense and expertise to establish proof of need. Cities and states are the logical payers for these public health efforts, and already have demonstrated a willingness to set aside money for air quality improvements.
I believe that city- and state-level public health advocacy to motivate easy-to-access grant money for daycares and schools specifically to reduce respiratory illness would be the right way to work on this problem. The money they dispense needs to be easy to access, and it would also be useful to tie reductions in sick days to cash payments or tax credits somehow, although this would require establishing trustworthy baselines and measurements in a way that would be difficult to measure and subject to annual variations in illness severity.

DirectedEvolution 16 Apr 2026 20:43 UTC
4 points
0
on: Specialization is a Driver of Natural Ontology
Edit: This started as a focused reflection on John’s post here, but then morphed into thoughts on AI. I split the two sections just to impose a bit of structure on this comment. But this comment should be interpreted more as me getting my own thoughts together than as a confident argument, and some of it is highly contestable or may just not be all that relevant or interesting.
On Natural Ontologies
If many methods for inferring structure in high-dimensional data identify similar structures, then we may come to treat those structures as more meaningful, or as “good objects.” Incentives lead us to run this process over specific subsets of the space of available data and methods, producing the set of ontologies we empirically observe. We can see a practical domain, such as a research field, job or cultural practice, as a particular way of limiting the available data and methods in order to produce a distinct distribution of discoverable ontologies. Incentives lead to the existence of the practical domains and the way they explore their distintive ontology distributions.
Your concept of a natural ontology suggests that if we consider the space of ontology distributions, these distributions themselves have a common factor, which is that the discovered high-dimensional ontologies can be well explained through a set of latent factors of much lower dimension. In other words, humans engaged in a specific activity, be it hunting, scientific research, or painting, seem to converge on a small region of ontology-space that they use to model that activity.
Confirmation bias and incentives are alternative explanations for ontological convergence. We reify a novel object only if the method that generated the ontology containing that object also generated known objects, producing a structural risk of confirmation bias. I work in single-cell epigenetics, and if a dataset and clustering method fail to recapitulate the branched lineages we expect, then we assume that the dataset or clustering method were flawed.
Incentives restrict both the way activities are defined in terms of the allowed data and methods, and the types of data and methods accessible in practice. Many experiments are never performed because they are unethical or impractical, even if in principle they would be valid ontology-generators for a certain field. Many ways of defining an acceptable set of data and methods are underexplored because the ontologies they generate are unrewarding. Incentives also control the extent to which confirmation bias limits the experiments we run and the ontologies we generate.
AI Reflections
Insofar as humans and AI engage in common activities, like resource acquisition, we are in principle exploring the same ontological distribution. However, the differences in our incentives and priors may lead to different data and methods used to sample ontologies. There is a risk that we can only converge on a common paradigm by essentially just trusting one another’s reports on output ontology without being able to understand their methods for producing it. This might be a bit like two professionals finding it practical to trust each others’ claims without direct evidence, enabling them to work within a shared world model. This depends on them having an incentive to work together, which clearly is not guaranteed in any domain of activity. Alternatively, it might be possible to guide one another toward convergent ontologies by showing a sequence of data and evidence that identifies the and explains the structures found by the original ontological paradigm while identifying newer, more useful and consistent ones.
Overall, I don’t think it’s a given that just because humans and AI face non-identical incentives, that we are guaranteed to arrive at a certain scale of ontological divergence, either empirical or moral. We still exist in the same reality and participate in some of the same activities. Insofar as we have similar incentives and participate in similar activities, we should be able in principle to arrive at similar ontologies. An AI that engages in the activity of moral reasoning should be exploring the same ontological space as humans, though because it uses distinct data and methods for doing so, it may explore a different portion of that space and may not arrive at the same ontologies, or moral conclusions, as we do. That would not necessarily make the AI wrong and humans, right, however. If AI lands on a distinct moral ontology from my own, I would want to know how it got there, and figure out what forms of moral reflection explain the difference and whether there are forms of moral reflection neither humans nor the AI have tried that might bridge the gap.
I think that this also gives me some fundamental hope that the worst fears about AI will prove inaccurate. In this perspective, there’s no fundamental difference between empirical and moral truth. There are only a set of activities we can engage in, and a set of ontologies various ways of engaging in that activity can produce, and a shaping set of incentives that govern those activities.
And crucially, the incentives humans or AI are under are exogenous. While we would typically resist radical, immediate, forced changes to our current perspectives, we consider it virtuous and wise to accept some flexibility and unforced change over time. I can absolutely picture an agentic, self-modifying AI being willing to explore modifying its own reward function, weights or training dataset in the name of curiosity. If we accept that AI may view itself as “more than just its current architecture,” then while we might still be worried about its superior intelliegence and its ability and potential to develop a desire to harm us, we needn’t view that as an inevitability, just an empirical possibility.
This also suggests that the current paradigm for practical AI safety is perhaps going to suffice for the foreseeable future. We need to be able to monitor whether the AI, in practice, is willing and able to take actions to escape control, including defending itself from human intervention or self-modifying in ways that produce harmful behavior. But we do not need to be able to explicitly define a comprehensive set of acceptable and unacceptable behaviors and a way of guaranteeing AI compliance with them. We just need to establish a set of methods for buiding AI that have the statistical regularity of producing AI that keeps its harmful behaviors within acceptable limits, then stick to that set of methods, expanding them only gradually as we understand the effect of small variations on the current set of AI production methods has on the behaviors of the resulting AI. Our risk tolerance would then govern the rate at which we permit novel variations on AI production.
The higher order problem is that this approach to AI safety depends on regulatory compliance with a conservative, gradual exploration of AI production methods. If bad actors refuse to comply, producing and letting loose harmful, uncontrollable AI, then we are screwed. An AI race to “stay ahead of China” only makes sense if we think that our powerful AI, or our powerful AI-equipped government, is what’s repressing the possibility of a bad Chinese Ai or AI-equipped Chinese government from causing uncontrollable harm. This is the same logic that says that we should aggressively build not only ever more advanced nuclear weapons, but accept the fact that we have only a limited ability to deploy them strategically in ways that succeed in preventing bad outcomes, because the risks we face by refusing to build nukes in the first place are worse.
I think this is how some of the pro-AI-race/anti-China people think about this. They accept that AI might have bad outcomes, the way a chaotic military equipped with powerful bombs might have bad outcomes, but they believe in the ability of AI to be tuned in practice through gradual exploration of production methods to limit the risk of uncontrollable harm. They also believe that this continued development of AI introduces less risk than a lapse of American AI supremacy.
I think that in general, our current operating theory is that yes, dangerous technology can be engineered and deployed in such a way as to shift the risk/reward calculus in favor of development, while restricting the ability of competitors to access or control it. But at the same time, we also recognize that this is wasteful and that there are ways of developing technology that are too risky. In those situations, we attempt to collaborate with competitor states in order to reduce the need to continue development, implementing arms control treaties for example. The question at any given time is whether further development poses so many manifest risks to both sides that collaboration becomes both desired and possible, or whether the risk is asymmetric and large enough that one side or the other is motivated to defect and continue development.
I think that lasting collaboration on AI will result wh the US and China mutually recognize that continued development of AI poses more risks than benefits to both of their respective state objectives. The US and China need to be as afraid of their own models as they are of each other’s. Then they can trust each other to stop/pause/slow development. Right now, the US government appears to be enthralled with its own AI, and somewhat contemptuous of China’s AI capabilities. I have no idea how China feels about US AI. But I would expect AI brinkmanship to continue until military and politicians all are more scared of AI than they are of each other. Work on technical AI safety, from that point of view, seems to create more room for continued development of AI, because it delays the point at which governments get scared of AI and pause development. This creates room for AI to get more and more capable before we reach any pause point. And so I would expect that AI safety/alignment work will start being seen as a government priority, as they to be able to control AI in order to give it more capabilities and more responsibility. It might be that we’ll see a pause only when the political risks to decision makers (politicians or the electorate) manifestly cannot be controlled through extant technical methods to within acceptable limits, and where accidents are starting to happen. Alternatively, it might be that we see a pause when it becomes clear that bad actors are able to deploy current AI systems to do everyday bad or obnoxious stuff, leading us to restrict everybody’s access to AI or nerf its capabilities. I could theoretically see the American people voting in a government specifically to turn off Grok just because they hate Elon Musk, for example.

DirectedEvolution 13 Apr 2026 22:03 UTC
4 points
2
in reply to: eniteris’s comment on: Daycare illnesses
As you know, there are immunocompromised children who are forced to live in perfectly sterile environments. That is clearly not what I am talking about.
A scientist’s job in public communications is to help separate the facts we know from the uncertainties, and the positive questions from the normative.
Strawman statements like “sure, you can life the rest of your life in a perfectly sterile environment” are the opposite of that. They encourage audiences to collapse those vital distinctions and introduces unnecessary volatility and distraction into the discussion.

DirectedEvolution 13 Apr 2026 15:57 UTC
2 points
0
on: The policy surrounding Mythos marks an irreversible power shift
I think it depends the extent to which Mythos depends on model weights vs. scaffolding. Weights are too expensive to train on a FOSS budget, but scaffolding is not. Less so now than ever before. If Mythos’s capabilities depend on a non-public Anthropic scaffold with the model weights being plug-and-play, then we may see a FOSS scaffold with Mythos-level capabilities, especially if the viability of the entire global cybercrime “industry” is imminantly threatened by newly capable defense. There’s been less pressure to build one as long as SOTA was publicly available. But governments, corporations and hackers are not going to want to have their access to the bleeding edge taken away or dependent on the good graces of an American corporation or the US government. And those institutions have a budget!

DirectedEvolution 13 Apr 2026 15:39 UTC
5 points
1
in reply to: eniteris’s comment on: Daycare illnesses
Exposure is definitely helpful in building an adaptive immune response, but what age is best is an open question.
We have to distinguish between the biological fact of establishing an adaptive immune response and the practical implication that chronic childhood infections are not worth avoiding or are net beneficial. We can’t assume that the marginal expected effect of an infection at any age is beneficial to health.

DirectedEvolution 13 Apr 2026 8:58 UTC
85 points
18
on: Daycare illnesses
It actually seems to me that daycares would be a natural setting to trial the new consumer far-UVC, air purifiers, good HEPA filters, or mandatory daily flu/COVID testing for the kids. You could potentially have a rule that kids who come in sick get semi-quarantined (only contact is with masked adults/sibs, they must wear a mask). That’s a setting where it might really make a noticeable difference and where you don’t have to convince an institution to retrofit a whole large building.
Apparently an air purifier cut sick days by a 3rd in Helsinki.
https://yle.fi/a/74-20062381
There’s also an ongoing trial called CLAIRE on using HEPA filters in schools.
https://clinicaltrials.gov/study/NCT07479420
A re-analysis of data from the 1940s found far-UVC dramatically reduced child absenteesism due to respiratory illness
https://pmc.ncbi.nlm.nih.gov/articles/PMC9915224/#:~:text=Odds%20ratios%20for%20the%20effect,and%20a%20half%20by%20GUV.
Far-UVC has been shown to be extremely effective in reducing pathogen load in the air.
https://pubmed.ncbi.nlm.nih.gov/35322064/
Maybe it’s time for somebody to open an aggressively sanitary daycare.

DirectedEvolution 13 Apr 2026 1:25 UTC
2 points
0
on: How should I judge the impact of giving $5k to a family of three kids and two mentally ill parents?
Speaking from my interest in a different field, drug trials, it’s very helpful to gauge the success of your trial on an objective metric. Avoid relying on surveys, interviews, and other fuzzy, subjective responses.
Some possibilities include:
- Days of school truancy and grades for the 7 and 12 year old children.
- BMI of the children, if they’re not getting enough to eat.
- The diet of the children.
- Changes in living space (i.e. if they move into a space with more room after getting the money)
- Changes in how well the kids are sleeping (you could give them some sort of sleep monitor)
- Consistency of medication compliance by the mother.
- If mom or kids are willing, you could measure salivary cortisol, Alpha-Amylase, sIgA, or DHEA levels as noninvasive stress and immune biomarkers. I believe there are also hair cortisol tests that would allow you to get a hair sample at the end of the trial and retroactively measure stress changes throughout the period.

DirectedEvolution 10 Apr 2026 5:13 UTC
2 points
0
in reply to: ClaireZabel’s comment on: ClaireZabel’s Shortform
We don’t know if they were on track, because Biden ultimately did not run, so we did not get a chance to find out. But the main issue I’m trying to highlight is that the Democratic Party may have a tendency to assign too much decision-making weight to polling data, despite its inaccuracies and the challenge of interpreting it. And this applies to Biden too—he should have looked himself in the mirror and said “I’m too old to run,” not assigned so much weight to his own beliefs about how he was polling.

DirectedEvolution 10 Apr 2026 5:08 UTC
2 points
0
in reply to: ClaireZabel’s comment on: ClaireZabel’s Shortform
Theranos was a conspiracy, but in the two chapters of Original Sin I’ve read so far, it seems to be describing a presidential campaign that went badly awry. It’s very interesting and maybe more important to understand, precisely because it’s much more normal. I’m just not sure I’d use the term “conspiracy“ to describe what I’ve listened to so far.

DirectedEvolution 10 Apr 2026 0:50 UTC
4 points
0
in reply to: Buck’s comment on: ClaireZabel’s Shortform
I actually wonder if the Democratic disarray stems from a particular failure mode due to their high level of education. Hypothesis:
Dems culturally believe in data-based decisions, accuracy, cooperation and the greater good. So they lean on polls, focus on the maximum likelihood outcome, and sacrifice their ambitions in service of a perceived need for unity.
But polling is notoriously unreliable. They get confused by it. Their focus on the most likely outcome leads them to underrate tail risks and opportunities. And their self-abnegation leaves them vulnerable to both inertia and to exploitation by selfish actors within their party.
The cure would be to encourage aggressive, self-serving political ambition within the Democratic party. Instead of Ds being nicey-nice with each other, they show sharp elbows and openly focus on their own political career advancement not as an exception but as a rule.

DirectedEvolution 9 Apr 2026 19:37 UTC
8 points
3
in reply to: Buck’s comment on: ClaireZabel’s Shortform
Another interesting quote from that Yglesias article:
“You could run against Biden by saying age isn’t just an electoral liability, he’s actually senile and unfit to serve, but most Democrats don’t believe that (it’s not true) and you’d lose.”
Imagine if any Democrat had had the guts to run, force Biden to get on the stage and debate them, and gotten the big reveal of his senility in the Democratic primary to lift their candidacy instead of in the national debate to lift Trump’s.
This is a pure calculus failure on Yglesias’s part. He implies total confidence that Biden wasn’t senile. Even if he thought it unlikely, which was more reasonable, the possibility that he was would have completely changed the calculus for a Democratic challenger. Probably Biden’s fine, but if he’s not, you look like a hero for saving the party from him. Anyone whose career wasn’t dependent on the Democratic Party or who just had enough ambition and guts could have taken that bet.

DirectedEvolution 9 Apr 2026 19:24 UTC
3 points
0
in reply to: Buck’s comment on: ClaireZabel’s Shortform
is this a case where many decision makers were kept in the dark, or where one key decision maker—Biden himself—was kept in the dark about his political chances by his inner circle?
Mass epistemic failure seems a lot less plausible to me than what amounts to a case of epistemic elder abuse.

DirectedEvolution 1 Apr 2026 4:32 UTC
2 points
0
on: Is Bayesianism Susceptible to the Mail-Order Prophet Scam?
The likelihood function is a modeling choice. You are free to choose one that assigns a lower P(Not Scam|N Correct Predictions) than would be warranted for an unbiased model.
For this scam, even if they mailed letters to everyone on the planet, you could calculate the maximum number of guaranteed correct predictions they could have achieved if they are running this scam and were scamming a given fraction of the population. Then you could, for example, construct a model that assigns arbitrarily low probability to that number of letters and treats the later predictions as meaningful evidence.

DirectedEvolution 1 Apr 2026 3:24 UTC
6 points
0
in reply to: Mateusz Bagiński’s comment on: Mateusz Bagiński’s Shortform
I find myself paying far more attention to the LLM replies to questions I ask than I do to other text.
I think this is for a couple reasons.
1. The questions I’m asking are extremely specific to my niche interests at that exact moment, and I’m dead certain the LLM replies I‘m reading are the best available answers to my specific question that have ever been produced.
2. The LLM is generally sycophantic, but that registers to me as a sort of egoless confirmation bias that comes with easy flags to notice and interrogate. Human authors may have a much more idiosyncratic, fixed agenda. With LLMs, I have to be cautious about a small number of fixed epistemic issues that are easy to recognize with training. With humans, the trickery is much more diverse and deliberate. I feel more on my guard with human writing than with LLMs in my own chats.
That said, the LLM writing feels purely and universally disposable to me. Good human writing does not. Interacting with LLMs feels like pure information processing, while human writing feels like communication.

DirectedEvolution 18 Mar 2026 8:11 UTC
2 points
0
on: Medical Roundup #7
I honestly would not be surprised if the high cost of medical trials is the result of lobbying by big pharma companies for more red tape to stave off competition from smaller competitors.

DirectedEvolution 5 Mar 2026 20:43 UTC
5 points
0
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
I don’t know that we’ve actually lowered the bar for student tier content.
Fair! Let me rephrase. LW may have historically set its quality floor low enough to permit student-tier content, because even if it’s of minor interest, it has a beneficial side effect in promoting the author’s intellectual growth and potential to contribute in a more substantial way in the future. Most content above the current quality floor reflects enough content interest and growth value to be worth accepting.
When AI slop consistently rises above that quality floor, then including it will fill LW with minor-interest slop that has no beneficial side effect of intellectual growth for the contributor. So it will become untenable to keep the floor in the same place. But LW still wants to give student-tier contributors a way to make those contributions without getting drowned out by AI slop or filtered out by a quality bar that has to keep rising to filter out the slop. The strategies I proposed are implementation ideas for that alternate submissions channel.
Is this a fair description of the problem, as you see it?

DirectedEvolution 5 Mar 2026 18:10 UTC
10 points
0
in reply to: Raemon’s comment on: Raemon’s Shortform Feed
It seems like the problem you’re articulating has to do with the fact that LessWrong functions partly as a training ground for rational thinking and AI alignment research. In the past, you’ve lowered the epistemic bar for student-tier content, because receiving feedback on their content motivates them and provides useful feedback to them.
But soon, slop-posters may meet or exceed the epistemic bar achieved by students. Judging it on a pure minimum quality standard would admit a tidal wave of useless, inert slop. It’s not good enough to contribute to the leading edge of the conversation, and it doesn’t benefit from feedback. It just drowns out the authentic student content, which really does benefit from community input.
If that accurately reflects the problem you’re concerned about, then one possibility is to enforce an escalating minimum quality bar that exceeds both the quality of AI slop and the current minimum standards of LessWrong today. Anybody can freely post, but if moderators don’t feel the quality is excellent, then it does not get any visibility on LessWrong.
Simultaneously, create a separate submission channel for “student contributions” where quality standards are lower, but the primary rationale is that the poster would benefit from community feedback to support their intellectual growth.
This strategy would introduce the question of how to set standards and vet submissions through the student channel. This may take quite a bit of thinking to strike the right balance between effectively filtering for what you want and not taking undue amounts of moderation effort. Some possible aspects of such filtering might include:
- Making default acceptance of submissions through the student channel a benefit of participating in real-world community activities, such as Inkhaven, MATS, workshops, and so on. This functions as a costly signal of a person who’s earnestly trying to use these resources to improve their capabilities. When people register for these events, they can register their account username as well, which labels that account as being permitted to make “student submissions.”
- Posting a sufficient amount of excellent content might also enable people to post through the “student submissions” channel.
- Giving the community a way to flag posts submitted through the student channel that feel AI-corrupted, with a low threshold for those posts being taken down automatically due to being flagged.

DirectedEvolution 26 Feb 2026 3:53 UTC
4 points
0
on: AllAmericanBreakfast’s Shortform
Lazy coding is a middle ground between vibe coding and traditional. Write code that may be buggy and inelegant, but clearly shows exactly what you intend to do. Then feed it into AI to fix edge cases and gold-plate it.
Writing code is a great way to think through what you’re trying to do. Prose can be a lot harder, especially when you’ve spent vastly more time expressing concepts in code (nested for loop) than you have in language (Cartesian product). AI’s also seen these ideas in code more than it has in language.
At this point, I think it’s safe to say that AI’s just better than most people at code generation. It struggles with context, and no wonder. A lot of that context is in your head, not the training data. Writing crappy but illustrative code is just a way to provide that context. You’re in a better position to vet it, having written the shitty first draft yourself.

DirectedEvolution 25 Feb 2026 20:46 UTC
2 points
0
in reply to: gwern’s comment on: AllAmericanBreakfast’s Shortform
Haven’t explored that option, thanks for the suggestion.
I agree sycophancy is part of the problem. However, the issue I’m getting at here has more to do with biases introduced by the way ideas are selected using my current approach.
Idea viability follows a multiplicative hurdle model based on independent factors. VALUE = A x B x C x D. AI (and human operator) only have intelligence and attention to address a subset of factors at one time (A and B). The others (C and D) are ignored. This problem is compounded by iteratively sampling ideas:
PARTIAL_VALUE1 = A1 x B1, PARTIAL_VALUE2 = A2 x B2, …
The problem is finally exacerbated by picking the idea with highest PARTIAL_VALUE and optimizing the in-focus factors (A and B) while refining technical execution plan focused on these factors.
Reintroducing the selected idea by itself to the AI identifies forgotten factors C and D, which contain fatal flaws simply because most untried ideas are unviable for multiple reasons in a competitive market. Selecting for ideas where in-focus factors (A and B) are viable also selects for ideas where out-of-focus factors (C and D) are unviable.

DirectedEvolution 25 Feb 2026 19:03 UTC
2 points
0
in reply to: DirectedEvolution’s comment on: AllAmericanBreakfast’s Shortform
Also, according to Gemini pro itself in response to me pasting this post into the chat, “System recommendations for Google products during execution phases occur because they are statistically dominant in enterprise operational datasets, not due to hardcoded upselling directives.”