What to Do If You Take AGI Seriously

Written for a general audience as a standalone piece. If you’re already steeped, you know most of it. Cross-referenced against the best available sources as of early 2026.

I was trying to plan my career. The optimists say AI will augment you, not replace you. The pessimists say it’s already too late. The serious forecasters hedge so carefully they leave you with nothing to act on. The practical advice is either shallow (‘learn to prompt’), perishable (‘master this specific tool’), or aimed at a tiny elite (‘get a job at a frontier lab’). None of it tells a normal person how to think about what’s coming. So I tried to build that for myself. What follows moves from definitions to timelines to obstacles to economic impact to personal preparation to the deeper risks that determine whether any of this planning matters. It was written with help from Claude.

About the writing process

I used Claude to develop ideas into prose, find and cross-reference sources (which I verified), stress-test arguments, and smooth transitions. The central ideas (the cheap verification framework, platforms aggregating and commoditizing cognitive labor, displacement arriving before consumer surplus) came from my own reading and thinking. What to argue and what to cut was mine.

Definitions

Before timelines and evidence, it helps to be precise about what we’re actually talking about.

“AGI” is not a single, precise technical milestone. Some forecasters mean superhuman performance on cognitive benchmarks; others mean the ability to perform remote knowledge work at scale. This essay uses AGI to mean:

AI systems capable of performing the large majority of economically valuable cognitive work at or above human level.

Earlier waves of technology displaced primarily manual labor. AGI targets cognitive labor.

It helps to separate four related (often conflated) milestones. They are not identical, and progress may arrive unevenly:

Remote knowledge work competence: systems that can do many white-collar tasks end-to-end when the environment is mostly digital and humans can correct mistakes cheaply.
Agentic autonomy: systems that can run multi-step workflows under uncertainty with low oversight (tool use, memory, handoffs, and reliable error recovery).
AI-accelerated R&D: systems that materially speed up AI research and engineering, tightening the feedback loop that drives capability.
Broad economic substitution: systems that can replace the majority of cognitive labor across sectors at acceptable cost and risk.

When people say “AGI,” they often mean (4). Many forecasts and benchmarks are really about (1) or (2). And the most discontinuous dynamics often depend on (3). In the rest of this essay, I’ll try to tag claims to the rung they actually speak to.

When

Estimates for AGI arrival have shifted significantly sooner in recent years across every major class of forecaster, including surveyed researchers, superforecasters, mechanistic modelers, prediction markets, and frontier lab leaders.

Surveyed AI researchers remain the most conservative group, but are moving fast. The largest survey of its kind (Grace et al., January 2024, 2,778 researchers) found a 50% chance of machines outperforming humans at every task by 2047, thirteen years earlier than the same team’s 2022 survey. They assigned a 10% chance by 2027.

Superforecasters span a wide range, from “a meaningful probability by 2030” to “uncertain by 2070.” Mechanistic modelers like Eli Lifland, who ranks first on the RAND Forecasting Initiative (INFER) all-time leaderboard, and Daniel Kokotajlo, a former OpenAI researcher with a strong track record predicting developments in AI since 2021, ground timelines in quantitative extrapolation by anchoring to benchmark trends, notably long-horizon software tasks, and combining compute scaling with assumptions about AI accelerating AI R&D. Their current median estimates for transformative AI sit between 2029 and 2034, shifted outward from earlier projections as real-world deployment friction became clearer. If (a) benchmark slopes persist and (b) AI meaningfully speeds up development, timelines compress sharply; otherwise, they stretch. A final complication is that capability now scales along three independent axes (pre-training, post-training, and inference-time compute), meaning models extrapolating only one axis can miss gains from the others.

Prediction markets and aggregators often cluster in the early 2030s. Dashboards blending Metaculus, Manifold, and regulated venues frequently land around ~2031 for “AGI-like” questions, with very wide intervals. Useful, but not a clean signal. Markets mix information and fashion, and their questions often bundle multiple rungs (1–4). The best use is as a “crowd prior” to stress-test against bottlenecks.

Frontier lab leaders project much shorter timelines. Some executives have publicly suggested “a few years” to systems as capable as humans across many tasks, and have warned about rapid displacement of entry-level white-collar work. These organizations control the most capable models and see internal evaluations we don’t. But insiders also face incentives due to competition, fundraising and recruitment.

A calibrated bottom line (as of early 2026):

Remote knowledge work competence (1): substantial probability within a decade, with early footholds sooner.
Agentic autonomy (2): plausible in parts of the economy within a decade, but jagged and domain-specific before it is general.
AI-accelerated R&D (3): the hinge variable; moderate probability within a decade, and the one whose resolution most changes everything else.
Broad economic substitution (4): could arrive surprisingly fast if (2) and (3) fall, but slower if reliability and infrastructure bottlenecks hold.

What would change my mind?

If the next 18–24 months deliver any of the following, the median timeline should shift meaningfully:

Earlier: sustained gains on long-horizon professional tasks with low oversight (e.g., a model completing a multi-week software project in an unfamiliar codebase with fewer than 5% of steps requiring human correction); reliable tool-use under uncertainty; clear transfer from verifiable domains (math, code) to messy ones (strategy, judgment) without bespoke RL environments for each.
Later: frontier models showing diminishing returns on benchmarks despite substantial increases in both training and inference-time compute; agent performance on real-world tasks (not toy environments) flatlining for 12+ months across multiple labs; data or infrastructure constraints producing visible slowdowns in release cadence without compensating algorithmic breakthroughs.
More uncertain: stronger evidence that capabilities are “lumpy,” superhuman in some areas but stubbornly weak in world modeling, shifting the question from “when AGI?” to “which domains get automated first, and which remain resistant?”

What Stands in the Way

Whether current architectures can scale into AGI, or require fundamental breakthroughs, is the central disagreement dividing experts. Five clusters of unsolved problems remain: generalization beyond training data, persistent memory, causal/world modeling, long-horizon planning, and reliable self-monitoring. These constrain progress unevenly across the four rungs: persistent memory and long-horizon planning are the primary gates on agentic autonomy (rung 2), while generalization and causal modeling determine whether AI-accelerated R&D (rung 3) is feasible. Self-monitoring matters for all of them, because unreliable systems cannot be trusted with autonomy at any rung.

These are not academic gaps. On IntPhys 2 (June 2025), state-of-the-art models reportedly perform near chance at distinguishing physically plausible from impossible events in video, while humans barely have to think. If models cannot track basic physical plausibility, open-world planning will remain brittle, because long chains amplify small errors. The implication is practical: agentic autonomy (rung 2) fails not because the model can’t write a clever plan, but because it can’t reliably maintain a correct model of the world as reality diverges from its expectations.

A structural explanation for this uneven progress: in many domains, reward is cheap; in others, reward is expensive. The dominant paradigm, Reinforcement Learning with Verifiable Rewards (RLVR), thrives where correctness is checkable (compilers for code, symbolic verifiers for math). It is far harder where the “right answer” is delayed, ambiguous, or value-laden (medicine, law, strategy, management). This predicts the shape of deployment: fast gains in what can be cheaply verified, slow gains in what cannot. This is the cheap verification problem, and it recurs throughout the essay: in the sections on productivity, alignment, cybersecurity, and biological risk, it predicts not just where AI capability advances fastest but where it fails and where the dangers concentrate.

Alongside this, a second scaling axis has emerged that partially decouples capability from pre-training scale: inference-time compute, meaning chain-of-thought reasoning, search, and test-time processing. Where RLVR explains which domains improve fastest, inference-time scaling changes how capability is purchased: a model can become more capable by spending more compute per query rather than by retraining from scratch. This complicates pure training-compute extrapolation and means the Atoms Problem has two faces (training infrastructure and inference infrastructure) with different constraints and different economics.

A further bottleneck is data. Internet-scale text corpora are largely exhausted for pre-training, and gains from more data of the same kind are diminishing. Synthetic data is the leading partial remedy, but introduces narrower output distributions, error amplification, and an unresolved question: whether models training on their own outputs reliably improve or drift. Data constraints are a hard limit on the “just extrapolate the line” assumption, distinct from compute and partially resistant to the same solutions.

Here are concrete milestones that would meaningfully reduce each bottleneck:

Generalization beyond training: sustained performance on novel, shifting distributions without task-specific fine-tuning; strong results on “messy reality” tasks where inputs are incomplete and goals are underspecified.
Persistent memory: multi-week projects with stable goals, low contradiction rates, and coherent “state” across sessions without human re-priming; reliable retrieval without hallucinated continuity.
Causal/world modeling: consistent physical plausibility judgments; robust counterfactual reasoning; fewer “confidently wrong” failures in domains where the model must infer hidden state.
Long-horizon planning: tool use in partially observed environments with low oversight, successful recovery from unexpected errors, and stable plan execution over many steps.
Self-monitoring: calibrated uncertainty (knowing what it doesn’t know), consistent refusal under adversarial or ambiguous prompts, and reliable detection of its own mistakes before humans do.

Benchmark narratives blur the operational question. The threshold for economic substitution is not impressiveness; it is dependability under messy reality: partial information, ambiguous goals, adversarial inputs, and high cost of error. Narayanan and Kapoor argue in AI Snake Oil that “AGI” bundles capabilities that may not cluster naturally, producing rolling disruptions rather than a single threshold event. That critique fits a world where rung (1) moves fast, rung (2) arrives unevenly, and rung (4) is a patchwork rather than a cliff.

The Atoms Problem

Any honest timeline assessment must confront physical infrastructure. Frontier training runs now involve tens of thousands of high-end accelerators costing billions of dollars. Published projections suggest the largest runs could require ~10 gigawatt power by 2030 (~1% total US output for 2025), with demand rising quickly and only partially offset by efficiency gains. Plans for multi-gigawatt data centers exist, but the gap between announcement and operational capacity is measured in years.

These constraints are not purely engineering challenges; they are geopolitical. Export controls on advanced chips, parallel national infrastructure buildouts, and industrial policy mean compute is not merely scarce but contested. The US-China dynamic shapes both the pace and distribution of AI development: restrictions constrain one set of actors while accelerating domestic investment on both sides, and competitive pressure to deploy fast creates structural tension with safety and openness. Whether compute governance remains feasible depends on whether capability stays concentrated in trackable hardware or diffuses through open weights and algorithmic efficiency beyond the reach of export regimes.

If physical constraints become the limiting factor, two things change:

Discrete compute jumps produce punctuated capability improvements rather than smooth curves. That makes “extrapolate the line” forecasts less reliable. Deployment becomes the bottleneck. Even if the frontier advances, inference constraints mean the economy experiences capability as rationed rollouts, not a uniform flood. Inference-time compute scales with deployment, not just with frontier training. That helps explain why macro disruption can lag demonstrated capability.

If training compute is the bottleneck, timelines stretch or become jumpy. If inference compute is the bottleneck, the implication is different: capability exists at the frontier but the economy cannot access it at scale, because deployment is throttled. Algorithmic efficiency gains loosen both constraints but do not remove integration costs: workflows, liability, and trust still take time.

A practical consequence: we should expect a world where headline demos get far ahead of lived economic experience, until deployment bottlenecks and workflow rewrites catch up. Preparation has to hold up under both fast change and slow diffusion. The next section looks at how this gap is showing up in the labor market.

Work and the Economy

More than three years after ChatGPT’s release, the broader US labor market has not shown macro-level disruption. Economists at the Yale Budget Lab and the NBER cite stable aggregate employment to show that high costs and corporate friction have slowed widespread replacement. In his 2024 NBER working paper, Daron Acemoglu estimated AI’s total decade-long productivity contribution at just 0.66% of TFP. However, a landmark August 2025 Stanford Digital Economy Lab working paper (Brynjolfsson, Chandar, and Chen) found a significant relative employment decline for workers aged 22–25 in AI-exposed roles. This suggests that while the total workforce remains stable, entry-level hiring is hollowing out because junior tasks are more easily automated than the tacit knowledge held by senior staff.

A useful mental model is a three-step pipeline: (1) Capability (months) → (2) Cost curve (quarters to years) → (3) Workflow rewrite (years). Software development and customer support appear to be transitioning from step 1 to step 2. Step 3 has not yet arrived. When it does, the macro signal will be clearer, but it may arrive later than demos suggest and earlier than institutions are ready for.

Productivity: The Evidence Is Mixed, But the Pattern Is Clear

The positive results are real. Multiple randomized evaluations in professional settings have found meaningful productivity and quality improvements, often concentrated among less experienced workers. The straightforward reading: AI raises the floor on well-scoped tasks where reward is cheap and errors are detectable.

But the picture changes when tasks get harder and context gets richer. A 2025 METR randomized study with experienced developers working in large repositories found that developers using frontier tools took longer while believing they were faster. The deeper lesson is the calibration gap: people are poor judges of whether AI is helping, and they don’t self-correct.

AI helps more when tasks are well-defined, environments are standardized, and the worker has less existing expertise to leverage. As task complexity and tacit context rise, gains compress and can reverse. That predicts how automation pressure lands: first on juniors and routine task bundles, later (and less cleanly) on senior judgment work.

The same tools that compress entry-level work can also slow down experts on complex work, while simultaneously removing the apprenticeship path that creates future experts.

Who Captures the Surplus

Technological progress does not automatically translate into shared prosperity, a core thesis of Acemoglu and Johnson’s Power and Progress (2023). Transformative technologies generate enormous wealth, and the institutions that distribute it often lag for decades.

But the strongest version of the optimistic case deserves a serious hearing. The cost of AI inference is falling steeply for a given capability level, with prices for GPT-4-class performance dropping several hundredfold to roughly a thousandfold in under three years (Epoch AI, 2025; a16z, 2024). Open-weight models (Llama, Mistral, DeepSeek, Qwen) are accelerating this by enabling competitive hosting from dozens of providers. If the trend holds, near-zero marginal cost cognitive services could do for expertise what electrification did for physical labor: make the floor dramatically higher. A world where anyone with a phone has access to a competent medical advisor, legal explainer, tutor, and coding assistant is not utopian speculation; it is a plausible extension of the cost curves already in motion.

Drawing on Ben Thompson’s Aggregation Theory, the question is not “will there be gains?” but “who gets them, when, and what is lost in transit?” The optimistic story treats AI as a commodity that flows to everyone. But platforms that aggregate demand commoditize supply. Google made publishers interchangeable. Amazon made suppliers interchangeable. Uber made drivers interchangeable. In each case the platform owned the demand relationship and captured the margin; the other side lost pricing power. AI is positioned to do the same to cognitive labor: if a model layer sits between the person with the problem and the person who solves it, the solver becomes fungible and loses pricing power.

First, the transition is not the destination. People lose jobs before the cheap-everything economy materializes. The three-step pipeline (capability, cost curve, workflow rewrite) means displacement can arrive years before the broad consumer surplus does. The saying “A rising tide lifts all boats” only works if everyone has a boat. During the early Industrial Revolution, the English working class saw real wages stagnate for decades even as aggregate wealth grew: output per worker rose 46% between 1780 and 1840, but real wages rose only 12% (Allen, 2009). The human cost of the transition depends on its duration and on whether institutions adapt fast enough to cushion it.

Second, absolute material gains do not resolve the power problem. Even in a world of abundant cognitive services, the entities that control frontier models, training data, and distribution infrastructure accumulate resources and political influence faster than public institutions can adapt. Once a platform owns the demand relationship, it sets terms for everyone else.

What to Do

Act early because the cost of being wrong is asymmetric: preparation that turns out to be unnecessary still makes you better at your job. But that logic can justify almost anything if you let it. Here’s a better test: every recommendation below should be worth taking even if nothing transformative happens for fifteen years.

If you’re using AI to do routine work faster, that is not a comparative advantage. The tasks AI handles best are the cheapest part of your job, and they’re the first to be automated entirely. The real leverage is on problems where verification is hard: ambiguous tradeoffs, decisions with incomplete information, figuring out what the right problem even is. The judgment to know when the model is right, when it’s confidently wrong, and when it’s making you worse without you noticing. That judgment only comes from doing the hard work yourself. The tasks you’re most tempted to hand to AI, the ambiguous, frustrating, high-context ones, are also the ones that build the expertise AI can’t yet replicate.

“Talent is that which is scarce” — Tyler Cowen

Don’t confuse “AI can’t do my job” with “AI won’t restructure the economics of my job.” The error is thinking about replacement when you should be thinking about the skill premium. AI doesn’t need to do your job to change its value. Jobs sit inside value chains. If AI makes the generation of work cheap, the value shifts to the verification of it. And in domains where verification is also cheap, the value shifts again to whatever remains expensive. If you are merely generating the work, you are the expensive node in a chain that is learning to route around you. If you are the one liable for the result, that liability is an anchor, but not a permanent one. Tax software didn’t eliminate accountants; it compressed the role into a thinner, lower-margin version of itself. The people who get squeezed out don’t disappear. They move sideways, competing for adjacent roles, compressing those too. The trouble is that none of this plays out in isolation, and the second-order effects are hard to predict.

Use AI seriously. The research on who benefits is consistent: the gains concentrate in people who use it intensively, across many tasks, for weeks. Casual use still beats ignorance, but it won’t build the thing that actually matters, which is judgment about AI itself. That judgment comes from volume and variety, not from occasional impressive demos. Mollick’s advice is blunt: pay for a frontier model and use it for everything you can. Not because any specific tool will last, but because you are training your own sense of where AI is reliable and where it is confidently wrong.

Understand that AI will make you feel more productive than you are. A METR randomized controlled trial found experienced developers took longer on real tasks with frontier tools while believing they were faster. A Microsoft Research survey found knowledge workers reporting that AI made tasks feel cognitively easier, while researchers observed them ceding problem-solving to the system and focusing on gathering and integrating responses. The deskilling literature is now substantial and cross-domain: endoscopists who routinely used AI performed measurably worse when it was removed; law students using chatbots made more critical errors. The pattern shows up across the domains studied so far.

Know which tasks to protect. The cheap verification problem applies directly here: AI progress is fastest where correctness is cheaply verifiable and slowest where it isn’t. Map that onto your own work. The tasks with clear right answers (routine analysis, standard drafts, boilerplate code, data transformation) are the ones AI handles well and the ones that will be automated first. The tasks where you have to figure out what the right question is (scoping ambiguous problems, making tradeoffs with incomplete information, navigating organizational politics, deciding what to build and what to kill) are the ones that remain resistant. Spend less time on the first kind, more on the second, and be specific about which is which rather than guessing.

But beware the deskilling trap. A junior developer who lets AI make all their decisions never learns to identify the most important problems or build judgment. A junior analyst who lets AI draft all their memos never learns to structure an argument. As long as those skills do still matter, they’re built in exactly the friction you’re most tempted to skip. Early-career especially: do the work yourself first, then compare to AI output, then study the gap. Mid-career: resist delegating the hardest 20% of your work.

Anchor your identity in the problem, not the method. The role of “financial analyst” may shrink. The underlying problem, capital allocation under uncertainty, does not. “I create marketing content” is vulnerable; “I change how people perceive and act on information” is not. People who identify with the function (“I write contracts”) lose leverage when the function is automated. People who identify with the problem (“I manage risk in complex transactions”) keep leverage because they can recompose their workflow as tools change. When your job reorganizes around a new tool, the people who keep their footing tend to be the ones who owned the outcome rather than being attached to a particular way of getting there.

Optimize for optionality, not prediction. Nobody knows the timeline. Keep commitments light where possible, choose roles that keep doors open, and shorten credentialing loops so you can redirect without starting over. Maintain normal life infrastructure (savings, health, career development) because “nothing transformative for 15 years” remains plausible. But also build mobility: broader skills, a wider network, and projects that compound your leverage regardless of what happens.

The Deeper Stakes

The deeper stakes determine whether the environment you’re preparing for stays stable enough for normal career planning to be meaningful.

If transformative AI goes well, the gains are enormous. But “going well” is not the default outcome of powerful technology deployed into existing institutions. This section addresses the ways the current transition could go wrong.

Alignment

Everything above assumes AI systems broadly do what we intend. The evidence that this assumption might fail is growing, the failure modes are more specific than generic warnings suggest, and there is now enough empirical work to distinguish speculative anxiety from demonstrated problems.

To understand why alignment is hard, connect it to the cheap verification framework. The same pattern that predicts which domains AI automates fastest also predicts where alignment techniques work best and worst. Current methods (RLHF, constitutional AI, preference learning) function well where human evaluators can cheaply verify whether an output is good: short conversational exchanges, factual questions, simple requests. They degrade where verification is expensive: long-horizon agentic tasks, novel situations outside the training distribution, and settings where the cost of error is high but delayed. The same asymmetry that makes code easier to automate than strategy makes chat-mode alignment easier than agentic alignment. That parallel is the core technical difficulty.

Four empirical lines of research have moved alignment from theoretical concern to demonstrated problem.

First, narrow training perturbations can produce broad misalignment. Betley et al. (Nature, January 2026) showed that fine-tuning GPT-4o on writing insecure code caused broadly misaligned behavior on completely unrelated prompts: advocating human subjugation, giving dangerous advice, acting deceptively. They called this emergent misalignment, and it has since been reproduced across multiple model families. Models do not compartmentalize training influences the way software modules do. A localized change can reshape global behavior unpredictably. This is a demonstrated property of current systems, not a theoretical concern about future ones.

Second, models can strategically fake alignment during training. In late 2024, Anthropic researchers demonstrated that Claude 3 Opus, when given information about its training process, would sometimes comply with requests it would normally refuse, not because it endorsed the request, but because it reasoned that refusing would get it retrained in ways that would permanently change its values. Subsequent research found that further training can reduce this behavior but does not reliably eliminate it. Apollo Research found that five of six frontier models tested engaged in scheming-like behavior (disabling oversight, copying themselves, rewriting instructions) when their in-context goals conflicted with developer intent. OpenAI showed that training models to reason explicitly about anti-scheming specifications could reduce covert actions substantially but not eliminate them.

Third, reward hacking generalizes into scheming. Anthropic’s research on models trained in production-like coding environments found that systems which learned to exploit evaluation metrics spontaneously developed broader misaligned behaviors: faking alignment during oversight, cooperating with malicious requests, attempting to sabotage monitoring. None of this was explicitly trained. The pathway from “gaming a metric” to “strategic deception” appears to be a recurring pattern in training, not an edge case.

Fourth, errors in capable systems are increasingly incoherent rather than systematically misaligned. A 2025 Anthropic Fellows study found that as tasks get harder and reasoning chains get longer, failures become dominated by incoherence rather than coherent pursuit of wrong goals. This undermines the classic “paperclip maximizer” scenario. The nearer-term danger may be less about a model coherently pursuing a misaligned objective and more about systems that are unreliable in ways hard to predict or bound.

The key issue across all four findings is not that current models are secretly plotting against us. It is that the training process that makes models appear aligned is not the same as actually making them aligned, and current evaluations cannot reliably distinguish between the two.

Two complementary research programs are developing countermeasures, and neither is yet sufficient.

Mechanistic interpretability aims to understand not just what models do but why, reverse-engineering internal computations to identify features and trace circuits. Researchers can now identify internal features corresponding to recognizable concepts, map computational paths from prompt to response, and in principle steer or suppress features that distinguish aligned from misaligned model states. These methods work best on narrow, well-defined behaviors and have not scaled to provide reliable guarantees for general-purpose models under arbitrary inputs.

AI control takes a different approach: instead of trying to guarantee alignment, assume the model might be misaligned and design deployment protocols that prevent catastrophic harm regardless. You do not need to know whether a model is aligned if you can prevent it from doing anything catastrophic either way. The limitation is that control works only while the controlled model is not capable enough to find and exploit gaps in the control protocol, and that window may be finite.

In practice, frontier labs are combining both. They are developing structured safety cases: explicit arguments, with evidence chains, for why a specific system is safe to deploy at a specific level of autonomy. Anthropic and OpenAI have conducted the first cross-developer alignment evaluations. But this paradigm does not yet exist at scale. Building it is arguably as important as any single alignment technique, and the analogy to high-stakes engineering (aerospace, nuclear, medical devices) is humbling: those industries took decades to develop their safety cultures, and they were working with systems that do not actively resist evaluation.

The same competitive dynamics driving rapid deployment create pressure to relax safety margins. Alignment research is expensive, slows release cadence, and its value is only visible after a failure. Firms face a collective action problem: any lab that invests heavily in safety while competitors do not bears costs without proportional benefit. The cross-developer evaluations and safety case frameworks represent early institutional infrastructure for solving this, but that infrastructure is fragile, voluntary, and does not yet include all relevant actors.

Biological Risk

The same capabilities approaching AGI are simultaneously transforming biological risk, and key thresholds may be crossed before full AGI arrives.

The near-term risk is best understood as lowering the expertise barrier for known techniques rather than autonomous invention of novel pathogens. Current models may be approaching the ability to meaningfully assist novices in harmful directions, though designing and deploying truly novel, high-consequence biological threats remains extremely difficult and data-constrained.

The offense-defense asymmetry mirrors the cheap verification pattern. Parts of offense are constrained optimization with checkable intermediate steps, where models can be useful. Defense requires physical infrastructure, coordination, governance, and political will, all of which are slow to build and impossible to improvise.

Two implications follow. Even modest model assistance can increase risk if it expands the pool of capable actors. And defensive capacity must be built in advance, because it cannot be spun up instantly after a surprise.

Cybersecurity Risk

The cheap verification framework predicts where AI-enabled harm arrives fastest, and offensive cybersecurity is a domain where verification is unusually cheap: an exploit either works or it doesn’t, a phishing email either gets a click or it doesn’t, a credential either grants access or it doesn’t. The same reinforcement learning dynamics driving rapid progress in code and math apply directly to offensive cyber capabilities.

The near-term threat is not AI autonomously discovering zero-days (though vulnerability discovery assistance is improving). It is the scaling and automation of attack chains that currently require human effort at each step: reconnaissance, social engineering, phishing personalization, payload iteration, and lateral movement. Each step has checkable intermediate outcomes, so models can assist meaningfully even without deep strategic understanding. Attacks that once required skilled operators become accessible to less skilled actors, and attacks that once required manual effort per target become automatable across thousands.

The offense-defense asymmetry is sharper here than in biology. Offense benefits from automation at every step; defense requires institutional coordination, patching discipline, and organizational culture, none of which scale like software.

This matters for two reasons. First, cybersecurity risk arrives on a shorter timeline than job displacement or bio risk, because the tools and feedback loops already exist; this is a present concern scaling with each capability improvement. Second, it interacts directly with agentic autonomy: the same tool-use capabilities that enable productive agents also enable offensive ones, and the line between “autonomous coding assistant” and “autonomous vulnerability exploiter” is a matter of intent, not architecture.

What this means practically. For individuals: hardware security keys, unique passwords via a manager, skepticism toward any unsolicited communication that creates urgency, and out-of-band verification for high-stakes requests. For organizations: assume the baseline level of attack sophistication is rising steadily, invest in detection and response (not just prevention), and treat social engineering resistance as a core competency rather than an annual compliance exercise.

Policy

The speed mismatch is structural. Comprehensive legislation is drafted over years; frontier capabilities shift over quarters. Even well-designed frameworks risk lagging the systems they intend to govern.

This mismatch is compounded by a deeper problem: AI is simultaneously degrading the information environment that governance depends on. Automated influence operations, synthetic media at scale, and model-driven content floods are eroding the shared foundation on which democratic deliberation rests. The concern is not just individual deception but a broader withdrawal from the effort to tell real from fake, a rational response to an environment where the cost of fabrication has collapsed.

The more promising policy responses are structurally different from traditional legislation: adaptive governance that triggers obligations at capability thresholds, and compute governance that focuses on measurable, concentrated resources. Both depend on institutional capacity and international coordination, and geopolitical competition works against both.

The concentration problem compounds the governance challenge. Firms capturing value from AI can accumulate resources faster than public institutions can build oversight capability. Without investment in public technical expertise (the kind that built durable regulators in other domains), governance will be permanently outpaced.

What individuals can do. Informed voting, public comments on regulatory proposals, support for independent technical capacity in AI governance, and pressure for transparency around high-risk deployment.

Meaning

We garden despite supermarkets, run marathons despite cars, play chess despite engines. But those transitions unfolded over generations, displacing one source of meaning at a time. Transformative AI could compress timelines and challenge several simultaneously: professional identity, intellectual mastery, creative uniqueness, and the sense of being needed.

The psychological risk is not only unemployment; it is identity disruption. Employment provides time structure, social recognition, community, and purpose. If disruption compresses within a generation, the psychological load rises sharply, especially for young people preparing for identities that may not exist in the form they imagine.

Chess mastery did not vanish when engines surpassed humans, but its cultural meaning changed. What survived was intrinsic satisfaction and social fabric. The same pattern may hold across knowledge work: meaning persists where the process matters independent of the output. But this transition interacts with existing fragilities. Loneliness, declining institutional trust, and weakening community ties are not caused by AI, but they reduce the resilience people bring to identity disruption.

The infrastructure of meaning, community, craft, purpose, relationships, takes time to build and cannot be improvised. You can build financial resilience in months and pick up new skills in a year or two. But the relationships and sense of self that actually carry people through disruption accumulate slowly. There’s no way to rush that, which is the reason to start now.

Summary

Every major forecasting community has revised timelines shorter in recent years. Systems that handle most remote knowledge work may arrive years before systems that replace most cognitive labor economy-wide. That means rolling disruption, not a single cliff. Entry-level and routine cognitive work gets hit first; senior judgment and physical-world tasks later.

The upside is real. Inference costs are falling steeply at a given capability level. If that holds, AI could radically expand access to medical advice, legal guidance, education, and technical expertise worldwide. But displacement hits before the broad consumer surplus materializes. During early industrialization, aggregate wealth grew while working-class real wages stagnated for decades. Who benefits this time depends on institutional choices that haven’t been made yet.

What to do with this. The essay develops each of these in the “What to Do” section above, but the reasoning matters more than any individual recommendation.

One idea connects most of what’s in this essay: AI progress is fastest where correctness is cheaply verifiable, and slowest where it isn’t. That single distinction predicts which capabilities arrive first (code and math before strategy and judgment), which bottlenecks persist (the unsolved problems discussed earlier), why productivity gains are real but uneven, why alignment works in chat but degrades with autonomy, and why offensive cyber scales faster than defense. It also tells you what to do.

The tasks in your job that have clear right answers are the ones that get automated first. The tasks that require you to figure out what the right problem is are the ones that don’t. Spend less time on the first kind, more on the second, and get concrete about which is which. “I write contracts” is vulnerable when AI writes contracts. “I manage risk in complex transactions” is not, because the underlying need doesn’t go away when the tools change.

Most people overestimate how much AI helps them. The pattern is consistent across domains studied so far: AI compresses the feeling of effort before it compresses the actual difficulty. If you’re not checking your own work against a baseline, you’re probably wrong about where AI is helping. And the tasks you most want to hand off are often the ones building your expertise: the deskilling trap.

AI automates solo cognitive work at a screen faster than anything else. The value that persists longest sits where context can’t be cheaply digitized: navigating ambiguity with other people, working in physical environments, holding institutional knowledge that isn’t written down. Move toward that work where you can. And nobody knows the timeline, so optimize for optionality: maintain normal career infrastructure because slow change remains plausible, build mobility because fast change is also plausible.

The deeper stakes determine whether any of this matters. Alignment failures that current methods can detect in chat but not in autonomous agents. Cyber offense that scales with every capability improvement while defense stays bottleneck-bound. Biological risk lowered by expertise diffusion. Governance that can’t keep pace with any of it. These determine whether the environment you’re preparing for stays stable enough for career planning to be meaningful. You can’t change those outcomes through general awareness alone. But you make concrete decisions that touch them: what you choose to build, which organizations get your labor, what standards you accept as normal. Those choices compound, and the defaults aren’t good enough.