I suspect you could get a lot of mileage just by separating out cyber and knowledge of AI monitoring, control, and alignment techniques. Although, cyber is tricky because it’s entangled with software engineering.
As a sanity check, I asked Claude (5.1 Fable), and it estimated that:
The portion of today’s economically valuable cognitive work that could be done without cyber and AI safety stuff is 90-97%.
The amount of pretraining tokens that would need to be filtered out to achieve ignorance of these topics is 5-20%, coming basically entirely from cyber.
An alternative approach to the low-knowledge agent missing things is to constantly run the knowledgeable agents in the background, with the option to submit requests to share information when they notice something.
Prompts and completions
Suppose an AI was just capable enough to automate all of today’s economically valuable cognitive work. Then suppose it had its knowledge of cybersecurity, AI monitoring, AI control, and AI alignment techniques removed. What percent of economically valuable cognitive work (as measured in dollars) could the resulting system automate?
Probably somewhere around 90–97%, with the uncertainty driven much more by how the removal works than by how much money those fields represent.
The direct hit is small. Cybersecurity is roughly a $200–300B global market, and a good chunk of that is hardware, licenses, and services rather than pure cognitive labor. AI alignment, control, and monitoring work is tiny by comparison—low single-digit billions in salaries at most. Against total global cognitive-labor compensation (on the order of $20–40T depending on how you draw the “cognitive” line), the fields you’ve named are on the order of 1% of the pie. So if the removal were surgical, you’d still automate ~99%.
The indirect hit is where the real uncertainty lives:
Entanglement. Security knowledge isn’t a separate compartment. Writing production software, running IT operations, configuring cloud infrastructure, doing legal/compliance work, auditing financials, designing payment systems, even drafting HR policies all embed security reasoning. A system that can’t think about threat models, input validation, access control, or adversarial users would produce work that fails review in a large fraction of software and operations roles—maybe 10–20% of the economy’s cognitive dollars touch this. It wouldn’t be unable to do those jobs, but output quality would drop below “automates the role” for some of them.
Fragility of “just capable enough.” You stipulated the system sits right at the threshold. Knowledge removal via unlearning or filtering typically causes collateral damage to adjacent capabilities (general reasoning about adversarial settings, systems thinking, some math). A system at threshold has no slack, so even a few-percent general degradation could push a broad set of tasks below the automation bar.
Re-acquisition. If the system has tools and internet access, “removed knowledge” is partly a fiction—it can look things up or reason from first principles. The realistic loss is then closer to the direct 1–3% figure. If it’s air-gapped from those resources, the loss is closer to the entanglement figure.
So: ~97% if the removal is clean and the system can reason or retrieve its way around gaps; ~90% (possibly lower) if the removal is blunt and the system is truly at threshold with no slack. The midpoint of ~93–95% is a reasonable single guess.
One honest caveat about the framing: the cost of the removal isn’t well captured by “percent of dollars.” The lost slice is disproportionately the work you’d most want done well—hardening the systems the AI itself runs on, and evaluating whether the AI itself is behaving. A 5% loss in dollar terms could be a much larger loss in what matters for deploying such a system safely at all.
Consider a frontier LLM training setup. Estimate the minimum percentage of the pretraining tokens that would need to be filtered in order to make the resulting model functionally ignorant of basic cybersecurity, as well as AI monitoring, control, and alignment techniques. Use your best guess based on public information.
The one real anchor
The best public data point is the Deep Ignorance study (EleutherAI + UK AISI, 2025), which is the only published work that reports actual filtering rates. Targeting biothreat-proxy knowledge in 6.9B models trained on 500B tokens, their strong filter removed 8.42% of pretraining documents and 9.36% of annealing documents, and they removed around 8–9% of the dataset while preserving general capability. Notably, 91.58% of pretraining documents didn’t even contain two blocklist terms — so the ~8% figure is mostly the cost of erring toward recall, not the true on-topic share, which is well under 1%. Deep Ignorance: Filtering Pretraining Data Builds +2
That gives a useful conversion factor: for a narrow, lexically distinctive, well-separated domain, “on-topic share <1%” turns into “~8–9% removed” once you calibrate for recall.
Why cyber is much worse than bio
Bio is close to the best case for filtering. “Pathogen,” “gain-of-function,” and “select agent” are rare words that rarely appear in unrelated text. Cybersecurity has the opposite property on every axis:
Lexical overlap is enormous. A two-keyword blocklist built from a WMDP-Cyber-style seed set (“buffer,” “authentication,” “exploit,” “root,” “packet,” “privilege,” “injection,” “hash,” “attacker”) would escalate something like a third of general web text and the large majority of code — versus 8% for bio. The blocklist stage does almost no work, so nearly everything rides on classifier precision.
The knowledge is not localized. Bioweapons knowledge lives in a specific literature. Basic security knowledge is a byproduct of ordinary material: SQL injection is in every web-dev tutorial, memory corruption falls out of any honest explanation of C pointers, TLS and password hashing are in every backend guide, phishing is in news stories and corporate emails and novels. You cannot filter the “security corpus” and be done.
It’s rederivable. A model that understands memory layout, parsers, and trust boundaries can reconstruct most of “basic cybersecurity” from first principles. Bio knowledge is much more empirical and less inferrable.
Explicit on-topic material (security blogs, CTF writeups, CVE/ExploitDB, USENIX Sec/CCS/arXiv cs.CR, offensive tooling repos) is probably 0.5–2% of tokens. But getting to functional ignorance means also cutting a large slice of systems programming, networking, sysadmin/DevOps, cryptography, and OS/browser internals — and if code is 15–20% of the mix, 20–40% of that code is security-load-bearing.
AI safety/control/alignment
Volume here is trivial — explicit alignment, interpretability, evals, and control content (arXiv safety papers, Alignment Forum/LessWrong, lab blogs, system cards) is maybe 0.01–0.1% of tokens. You could delete all of it for well under 1% and barely dent anything.
The problem is that it doesn’t buy ignorance:
Entanglement with ML generally. RLHF, red-teaming, and evals are inseparable from the ML corpus. Robust ignorance means cutting most of cs.LG, which costs a few percent and cripples the model at ML work.
Self-evidence. A model trained with RLHF or a constitution is being shown, in-distribution, that its outputs are being graded. Post-training reintroduces the concept regardless of what pretraining saw.
In-context leakage is the killer. Deep Ignorance found that filtered models lack internalized knowledge but can still leverage the information when it’s provided in context, e.g. via search tools. A deployed agent reads system prompts, tool schemas, monitoring output, and refusal messages. Control techniques announce themselves at inference time in a way bioweapons synthesis routes do not. arXiv
The estimate
Goal
Approx. tokens filtered
Explicit on-topic content only (leaky; benchmark scores drop, real knowledge persists)
1–3%
Functional ignorance on cyber benchmarks + alignment/control literature
8–15%
Robust ignorance surviving adversarial elicitation and rederivation
25%+, and probably not achievable
My central guess is ~10%, with a plausible range of 5–20% — with the split being roughly 8–14 points for cybersecurity and under 1 point for the alignment/control material.
Three caveats worth more than the number itself. First, the binding constraint is separability, not volume: the AI-safety half is nearly free to remove and nearly useless to remove, while the cyber half is expensive and only partially effective. Second, the capability tax is concentrated exactly where frontier labs make money — a model missing systems programming and networking is a much worse coding agent, which is likely why no lab does this for cyber even though several report bio filtering. Third, “functionally ignorant” isn’t well-defined without specifying the threat model; a model that fails WMDP-Cyber but can still reason from a man page in context hasn’t been made safe, just made to score lower.
I suspect you could get a lot of mileage just by separating out cyber and knowledge of AI monitoring, control, and alignment techniques. Although, cyber is tricky because it’s entangled with software engineering.
As a sanity check, I asked Claude (5.1 Fable), and it estimated that:
The portion of today’s economically valuable cognitive work that could be done without cyber and AI safety stuff is 90-97%.
The amount of pretraining tokens that would need to be filtered out to achieve ignorance of these topics is 5-20%, coming basically entirely from cyber.
An alternative approach to the low-knowledge agent missing things is to constantly run the knowledgeable agents in the background, with the option to submit requests to share information when they notice something.
Prompts and completions
Suppose an AI was just capable enough to automate all of today’s economically valuable cognitive work. Then suppose it had its knowledge of cybersecurity, AI monitoring, AI control, and AI alignment techniques removed. What percent of economically valuable cognitive work (as measured in dollars) could the resulting system automate?
Probably somewhere around 90–97%, with the uncertainty driven much more by how the removal works than by how much money those fields represent.
The direct hit is small. Cybersecurity is roughly a $200–300B global market, and a good chunk of that is hardware, licenses, and services rather than pure cognitive labor. AI alignment, control, and monitoring work is tiny by comparison—low single-digit billions in salaries at most. Against total global cognitive-labor compensation (on the order of $20–40T depending on how you draw the “cognitive” line), the fields you’ve named are on the order of 1% of the pie. So if the removal were surgical, you’d still automate ~99%.
The indirect hit is where the real uncertainty lives:
Entanglement. Security knowledge isn’t a separate compartment. Writing production software, running IT operations, configuring cloud infrastructure, doing legal/compliance work, auditing financials, designing payment systems, even drafting HR policies all embed security reasoning. A system that can’t think about threat models, input validation, access control, or adversarial users would produce work that fails review in a large fraction of software and operations roles—maybe 10–20% of the economy’s cognitive dollars touch this. It wouldn’t be unable to do those jobs, but output quality would drop below “automates the role” for some of them.
Fragility of “just capable enough.” You stipulated the system sits right at the threshold. Knowledge removal via unlearning or filtering typically causes collateral damage to adjacent capabilities (general reasoning about adversarial settings, systems thinking, some math). A system at threshold has no slack, so even a few-percent general degradation could push a broad set of tasks below the automation bar.
Re-acquisition. If the system has tools and internet access, “removed knowledge” is partly a fiction—it can look things up or reason from first principles. The realistic loss is then closer to the direct 1–3% figure. If it’s air-gapped from those resources, the loss is closer to the entanglement figure.
So: ~97% if the removal is clean and the system can reason or retrieve its way around gaps; ~90% (possibly lower) if the removal is blunt and the system is truly at threshold with no slack. The midpoint of ~93–95% is a reasonable single guess.
One honest caveat about the framing: the cost of the removal isn’t well captured by “percent of dollars.” The lost slice is disproportionately the work you’d most want done well—hardening the systems the AI itself runs on, and evaluating whether the AI itself is behaving. A 5% loss in dollar terms could be a much larger loss in what matters for deploying such a system safely at all.
Consider a frontier LLM training setup. Estimate the minimum percentage of the pretraining tokens that would need to be filtered in order to make the resulting model functionally ignorant of basic cybersecurity, as well as AI monitoring, control, and alignment techniques. Use your best guess based on public information.
The one real anchor
The best public data point is the Deep Ignorance study (EleutherAI + UK AISI, 2025), which is the only published work that reports actual filtering rates. Targeting biothreat-proxy knowledge in 6.9B models trained on 500B tokens, their strong filter removed 8.42% of pretraining documents and 9.36% of annealing documents, and they removed around 8–9% of the dataset while preserving general capability. Notably, 91.58% of pretraining documents didn’t even contain two blocklist terms — so the ~8% figure is mostly the cost of erring toward recall, not the true on-topic share, which is well under 1%. Deep Ignorance: Filtering Pretraining Data Builds +2
That gives a useful conversion factor: for a narrow, lexically distinctive, well-separated domain, “on-topic share <1%” turns into “~8–9% removed” once you calibrate for recall.
Why cyber is much worse than bio
Bio is close to the best case for filtering. “Pathogen,” “gain-of-function,” and “select agent” are rare words that rarely appear in unrelated text. Cybersecurity has the opposite property on every axis:
Lexical overlap is enormous. A two-keyword blocklist built from a WMDP-Cyber-style seed set (“buffer,” “authentication,” “exploit,” “root,” “packet,” “privilege,” “injection,” “hash,” “attacker”) would escalate something like a third of general web text and the large majority of code — versus 8% for bio. The blocklist stage does almost no work, so nearly everything rides on classifier precision.
The knowledge is not localized. Bioweapons knowledge lives in a specific literature. Basic security knowledge is a byproduct of ordinary material: SQL injection is in every web-dev tutorial, memory corruption falls out of any honest explanation of C pointers, TLS and password hashing are in every backend guide, phishing is in news stories and corporate emails and novels. You cannot filter the “security corpus” and be done.
It’s rederivable. A model that understands memory layout, parsers, and trust boundaries can reconstruct most of “basic cybersecurity” from first principles. Bio knowledge is much more empirical and less inferrable.
Explicit on-topic material (security blogs, CTF writeups, CVE/ExploitDB, USENIX Sec/CCS/arXiv cs.CR, offensive tooling repos) is probably 0.5–2% of tokens. But getting to functional ignorance means also cutting a large slice of systems programming, networking, sysadmin/DevOps, cryptography, and OS/browser internals — and if code is 15–20% of the mix, 20–40% of that code is security-load-bearing.
AI safety/control/alignment
Volume here is trivial — explicit alignment, interpretability, evals, and control content (arXiv safety papers, Alignment Forum/LessWrong, lab blogs, system cards) is maybe 0.01–0.1% of tokens. You could delete all of it for well under 1% and barely dent anything.
The problem is that it doesn’t buy ignorance:
Entanglement with ML generally. RLHF, red-teaming, and evals are inseparable from the ML corpus. Robust ignorance means cutting most of cs.LG, which costs a few percent and cripples the model at ML work.
Self-evidence. A model trained with RLHF or a constitution is being shown, in-distribution, that its outputs are being graded. Post-training reintroduces the concept regardless of what pretraining saw.
In-context leakage is the killer. Deep Ignorance found that filtered models lack internalized knowledge but can still leverage the information when it’s provided in context, e.g. via search tools. A deployed agent reads system prompts, tool schemas, monitoring output, and refusal messages. Control techniques announce themselves at inference time in a way bioweapons synthesis routes do not. arXiv
The estimate
Goal
Approx. tokens filtered
Explicit on-topic content only (leaky; benchmark scores drop, real knowledge persists)
1–3%
Functional ignorance on cyber benchmarks + alignment/control literature
8–15%
Robust ignorance surviving adversarial elicitation and rederivation
25%+, and probably not achievable
My central guess is ~10%, with a plausible range of 5–20% — with the split being roughly 8–14 points for cybersecurity and under 1 point for the alignment/control material.
Three caveats worth more than the number itself. First, the binding constraint is separability, not volume: the AI-safety half is nearly free to remove and nearly useless to remove, while the cyber half is expensive and only partially effective. Second, the capability tax is concentrated exactly where frontier labs make money — a model missing systems programming and networking is a much worse coding agent, which is likely why no lab does this for cyber even though several report bio filtering. Third, “functionally ignorant” isn’t well-defined without specifying the threat model; a model that fails WMDP-Cyber but can still reason from a man page in context hasn’t been made safe, just made to score lower.