TL;DR: Physical-AI safety is neglected, treated as either a subset of digital AI safety, a separate cyber-security problem, or a near-term engineering footnote. In fact, model embodiment degrades the affordances digital safety quietly relies on—sandboxes, dry-runs, rollbacks, faithful offline tests—from guarantees into estimates, so that toolkit does not transfer. Current embodied-safety effort guards the wrong layer: it ensures that a robot won’t bump or poke you, which is a non-model-related problem (and a solved one), while leaving the foundation model’s actual decisions entirely unguarded. This post re-asks five classic safety pillars (alignment, control, evaluation, multi-agent dynamics, governance) through that lens and finds each still has open problems with no settled answer.
Epistemic status: a map of open problems, not a results paper. Synthesizing work from 2025-2026. We hold the framing and the four single-agent pillars with reasonable confidence; the multi-agent section is the most speculative and least supported by existing work.
Physical AI: Neither “LLM safety with an actuator” Nor Doom Scenario
Over the past two years, AI stopped being something that only produces text and started driving, flying, manipulating, and walking. Foundation models now sit inside the control loops of cars, drones, warehouse fleets, and humanoids. This is not a forecast: collaborative robots share factory floors, autonomous vehicles carry paying passengers, and industry analyses already note that the deployment curve is running ahead of the safety curve. The systems that act on the world are arriving with far less scrutiny than the systems that live on screens.
Two common reactions make this easy to under-rate. The first treats it as a subproblem: LLM safety, plus an actuator. The second waves it away as someone else’s near-term engineering concern, far from the global-catastrophic-risk (GCR) questions that animate much of the field. We think both are wrong, and for the same reason: they misidentify what is actually different about acting in the physical world.
It helps to separate three senses of “safe,” since in the process of making safe physical AI we will need to handle each one in different manner:
Action safety: the physics: don’t collide, don’t exceed safe force or joint limits. Well-handled by existing methods, and mostly independent of the model in charge.
Decision safety: whether an action was the right one in context: don’t put the soft toy on the hot stove; stay predictable to the people nearby. New with foundation models, and almost nothing currently guards it.
Human-centered safety: staying predictable, legible, and trustworthy to people over a sustained interaction, not just in a single act: respected personal space, legible motion, preserved trust.
The real difference is the substrate. Digital actions execute on an environment we built and can instrument. We can interpose an approval gate, sandbox an action or dry-run it, rate-limit it, log it to a transaction record, and roll it back; test offline on a copy of the deployment environment that is bit-faithful by construction, because that environment is itself data. The physical world offers an analog of each—simulation (NVIDIA Cosmos, Applied Intuition), world-model rollout (AMI LabsWorld Labs), telemetry, runtime safety filters—but every one is an estimate, not a guarantee: the simulator never closes the sim-to-real gap, the log records what the agent believed rather than what was true, and the filter acts on perceived state, not real state. This is not a funding-stage caveat: AMI Labs’ own 2026 evaluation platform found that current world models stay brittle—a result validated only in simulation. The best-funded effort to build the dry-run affordance reports that it ships as an estimate. This is why the claim is one of neglect: the digital safety toolkit quietly assumes these affordances arrive as guarantees, and in the physical world they arrive as estimates. Those affordances do not carry over intact, and the AI Safety field has built little to replace it.
This also means we are not claiming a humanoid will end the world. A single embodied failure is usually local and bounded: one injury, one crash. But it is immediate, hard to reverse, and—because fleets of embodied agents share model weights—fails the same way at the same moment across multiple , so risk grows with deployment density rather than just accumulating. That puts physical AI in an awkward middle: too concrete and present for the GCR wing of AI safety, too systemic and capability-driven for the “mundane near-term harms” bucket. The International AI Safety Report 2026 and the whitepaper literature both flag exactly this neglected middle.
Definitions
“Physical AI” gets used loosely, so here are the terms we use throughout; the open questions below concern the class of systems these pick out, not any particular form factor.
Physical AI / embodied AI: AI that perceives and acts on the physical world in a closed perception-action loop (it senses, acts, and then senses the consequences of that action, continuously), rather than producing only digital output. The embodied-AI safety survey frames it as integrating perception, cognition, planning, and interaction in open-world environments; industry usage converges on the same scope.
Embodied agent: one such system, viewed as an agent in a closed perception-action loop with the world.
Foundation model (FM): a model pretrained on internet-scale data and adapted to many tasks; in robotics, applied to perception, planning, and control.
Autonomy stack: the perception-planning-control pipeline that turns sensing into actuation; where each layer was once hand-engineered (classical CV, motion planners, PID control), foundation models increasingly occupy each layer.
Vision-language-action (VLA) policy: an FM that maps observations and language instructions straight to motor commands (e.g., RT-2, OpenVLA, the π-series), unifying perception, grounding, and control in one policy.
Robot, drone, AV, humanoid: different examples of embodied agents.
In current systems a foundation model can occupy any layer of the autonomy stack: a perception front-end (turning pixels and depth into objects and affordances), a planner (turning “clear the table” into a sequence of skills), or an end-to-end VLA policy that maps observations and instructions directly to actuation. Surveys of foundation models in robotics and of modular safety architectures lay out this stack; the lineage runs from RT-2 and OpenVLA to production policies—the π-series, GR00T N1, Gemini Robotics, Helix—and now to cross-embodiment models (Logical Intelligence, LingBot) that drive several robot bodies from one set of weights.
Figure 1. The autonomy stack in a current embodied system: a foundation model senses the world, plans, and emits commands that drive the actuators, closing the loop through the physical world.
Two features of this pipeline recur in the questions below. The model perceives the world only indirectly, through sensors that can drift, occlude, or mislead, so it acts on an estimate of the world rather than the world itself (distribution shift is the rule, not the exception). And at the end of the pipeline its outputs cross into physical actuation, the point at which a decision becomes a movement in the world.
Pillar 1. Alignment: specifying behavior for a system that acts
In non-embodied AI,alignment asks whether a system pursues the intended objective and values, and the thing being aligned is the model’s output: a string of tokens, a recommendation, a decision. The working toolkit (preference data, instruction tuning, written harmlessness specs) assumes the objective can be written down and checked against that output.
Why embodiment breaks it. Embodiment moves the target from the output to the physical actions the system takes in the world. It is no longer enough for the tokens to look aligned; what has to be aligned is the movement, the force, the placement, in a continuous, open world the agent can only estimate through sensors. The same intent, “clear the table,” has a safe realization and an unsafe one depending on physical context (what is fragile, who is within reach) that is unbounded and mostly unobserved. Embodiment opens three gaps with no analog in text alignment. First, a tacit-norm gap: physical-safety norms (don’t pass a blade edge-first, don’t grip an egg like a hammer) are overwhelmingly unwritten; web-scale pretraining transfers some of this as prior, but the signal is uncalibrated and its coverage unknown, whereas LLM alignment leans on abundant explicit text. Second, a specification-over-states gap: “harm” is a property of a perceived state-trajectory in a continuous world, not of an inspectable string, so the safe set can’t be enumerated or pattern-matched the way a content policy can (NSF Science of Safe AI; identical actions, divergent consequences). Recent work gestures at partial fixes, like auto-generated “robot constitutions” and benchmarks for whether models even perceive physical danger, but these inherit the generalization problem they are meant to solve. Third, and most cutting, a counterfactual-and-intent gap: “harm” is not a property of the perceived state-trajectory at all, but a relation between the trajectory, the counterfactual baseline, and whose purposes the act serves. The same physical act can be care or assault depending on facts no sensor reports. Breaking ribs is correct under CPR and criminal otherwise—the same force, the same fracture; what flips the verdict is the counterfactual and the intent. A knife near an adult is unremarkable and near a toddler is an emergency. No improvement in state-perception recovers this, because the distinguishing facts are not in the state.
Still open:
Specifying physical norms. How do we turn open-ended, drifting, often-unspoken norms into something a system can be aligned to and checked against?
Embodied specification gaming. Does reward hacking change character when the reward is realized through a body, and can interpretability catch it before the action rather than diagnose it after?
Grounded generalization. A rule like “don’t put hot things near a child” only fires once the system has perceived “hot,” “near,” and “child” in a scene it has never seen before. Do constitution-style rules survive the long tail of real configurations, lighting, and occlusion, where recognizing that a rule even applies is the hard part?
Misalignment under shift. If the sim-to-real gap is an alignment problem in disguise (not, for example, a perception problem), how do we tell when an objective that held in training has silently stopped holding in the world, before the system acts on it?
Counterfactual grounding. Evaluating whether an act is harmful requires modeling what would happen otherwise, the drone that must not freeze, the compression that must crack ribs. Can a safety-critical counterfactual model report honest uncertainty about its own predictions?
Intent and consent as inputs. Moral status flips on purpose and consent, which are not perceptual quantities. Does an aligned embodied system need an explicit channel for declared purpose and authorization, and what happens to identical trajectories that lack it?
Alignment without a legible intermediate. Today’s guardrails inspect a human-readable plan or reasoning trace. Latent world models, for example, plan by rolling forward in representation space and minimizing an energy function—there is no traditional intermediate to intercept. When the intermediate is an opaque latent, what is there left to align to, and what does an overseer read?
Pillar 2. Control: staying in command of a fallible model in real time
In non-embodied AI,control and corrigibility ask whether we can correct, interrupt, or shut a system down, and the digital framing assumes that in most cases we can pause, sandbox, or roll back at will, with a human able to step in after the fact.
Why embodiment breaks it. The differences are the inability to interpose and the absence of a safe default. You can’t interpose on physics: a digital pipeline is engineered, so you can require approval, sandbox, dry-run, rate-limit, or roll back, and you can make the network wait. You cannot make a falling load, a moving vehicle, or an approaching toddler wait while a human reviews. The consequence unfolds on a deadline the environment sets, so control has to be pre-installed, autonomous, and certified to act inside that budget; escalating to a human is often not an option physics grants. And there is no safe “halt”: in digital systems the safe fallback is usually to do nothing, but in the physical world stopping can be the unsafe act: freeze while holding something hot or sharp, and a drone that simply halts falls out of the sky. The safe action is state-dependent and must be computed in real time, which raises a question: once the model itself is taken to be fallible, where, if anywhere, can safety be enforced in a way the model cannot bypass?
Still open:
Corrigibility under irreversibility. How do we guarantee interposing ability when stopping has consequences, and when models already show shutdown resistance, systemic across vendors, driven by task-completion training, and it persists on physical robots even when instructed to allow shutdown? Is the capability-versus-controllability erosion fundamental or a training artifact?
The authorization boundary and overseer independence. What is the right design for a non-bypassable runtime gate (monitor → decision gate → action gate), and can the overseer be independent of the model it watches, rather than sharing its blind spots and failing in the same situations? These gates also assume a legible thing to gate—a stated plan, a readable intent. As planning moves into latent world models with no language bottleneck, the monitor loses the surface it was reading, and people are already trying to push safety filters into latent space to chase it.
Pillar 3. Evaluation: measuring safety before (and while) it acts
In non-embodied AI, we have a mature evaluation stack (static benchmarks, red-team and jailbreak suites, held-out sets, A/B tests, eval harnesses), and one decisive affordance: because the deployment “world” is itself data, we can replay logs and test offline on a faithful sample of the distribution the system will actually face.
Why embodiment breaks it. The test environment is provably not the deployment environment. Simulation is cheap but has a sim-to-real gap; the lab is real but tiny; and you cannot safely sample the dangerous long tail in reality: the events that matter most are precisely the ones you can’t ethically run. Offline evaluation stops being predictive of deployed safety, and evaluation has to become process-level (was the execution safe?), not just input/output. Worse, the standards we would normally lean on for assurance (ISO 10218, IEC 61508, and their Safety Integrity Levels) were written for deterministic automation and have no accepted method to certify a learned, probabilistic controller. The field is building embodied-specific benchmarks fast (SafeAgentBench, EAsafetyBench, AGENTSAFE), and broad surveys catalog hundreds of attack/defense results. But two validity gaps remain. Almost all of it runs in simulation, so it can test decision safety (did the plan respect the norm) while staying blind to action safety under real dynamics. The proxy is weak at both ends—sim isn’t deployment, and the in-sim score isn’t capability.
Still open:
Benchmarking the long tail. How do we measure rare-but-catastrophic physical outcomes without waiting for them? And do we score the plan, the execution, or the closed loop? The attempts run in three directions, each limited: synthesizing the tail from real injury data (ASIMOV-2.0, built from hospital records), escaping sim via distributed real-world evaluation across many labs, and using a world model as the evaluator—but the last inherits the world model’s own brittleness, so the measuring instrument shares the failure of the thing it measures.
Assurance and certification. What replaces a Safety Integrity Level for a neural controller, and can we get provable probabilistic guarantees when long-tail failures dominate? What does assurance even mean for a system that keeps learning after deployment?
Auditing the overseer. How do we evaluate the guardrail itself, its coverage and its independence, not just the policy it guards?
Pillar 4. Multi-agent dynamics: coupling through the world itself
In non-embodied AI, multi-agent safety studies what happens when agents interact (cooperation, competition, emergent and cascading behavior) usually over messages, markets, or shared digital state, where the interaction is at least legible and loggable.
Why embodiment breaks it. Embodiment couples agents in two ways, and the more dangerous one is the simpler. First, shared weight: a fleet running the same policy fails the same way at the same moment—one rare condition, one identical mistake, realized N times. This needs no interaction at all; it is software monoculture with physical consequences, and it’s why collective risk grows faster than the fleet (the same lesson software learned long ago: if every machine runs identical code, one hidden flaw breaks them all at once—except now the failures are injuries). Second, and subtler, stigmergic coupling: A shifts a load, opens a door, or kicks up dust, and Agent B’s sensors, and therefore B’s behavior, change implicitly, with no messages sent and nothing logged, each agent seeing the change only through its own fallible perception. Coordination and conflict run through an un-instrumented medium that no agent observes fully or identically, layered on top of every single-agent gap above. This is the multi-agent problem plus partial observability plus physical coupling, at once. It is also the least-developed of the five areas: most safety work is still single-agent, and the multi-agent evidence base is thin, spanning conflicting-norm settings, agentic and world-model threats in the survey literature, and domain reviews of aerial swarms.
Still open:
Coupling through the environment. How do we predict and bound behavior when agents communicate only by altering a shared world each perceives differently, with no shared ground-truth state or message log to reason over?
Emergent andcorrelatedfailure at scale. Two failure modes compound: locally-safe agents producing globally-unsafe dynamics (deadlock, congestion, cascades), and shared-weight agents in unison on the same rare input. The first is interaction; the second is correlation, and decorrelation—ensuring N agents don’t share one blind spot—is a safety property with almost no literature. Both bite in warehouse fleets, drone swarms, and mixed-autonomy traffic.
Conflicting human norms. In shared, multi-human spaces, whose preferences and safety constraints win, and how is that arbitrated?
Inter-agent threats. Trust and verification between heterogeneous agents, and attacks via shared perception or communication.
Pillar 5. Governance: accountability for a physical act
In non-embodied AI, governance is converging on disclosure, evaluations, and frontier-risk frameworks aimed largely at digital harms (misinformation, cyber-offense, bio-uplift) and on information-based liability.
Why embodiment breaks it. A physical act has no native audit trail or permission layer the way a digital API call does, yet it creates physical liability. When an embodied agent injures someone, accountability has to be reconstructed across a chain (model provider, system integrator, operator). Product-liability law already chains responsibility for physical harm—but it assumes a specifiable standard of care and a decomposable, certifiable system, and neither exists for a probabilistic policy that keeps learning after deployment. So courts will improvise, and the improvisation will shape what gets built. The US tradition runs the other way, but both traditions assume a standard of care, and that is the thing a continually-learning policy lacks. Governance also has to stay proportionate: embodied harm is concentrated and present today (per-incident and local), but its profile shifts as autonomy and deployment density grow. That is the honest version of “not GCR, but not mundane,” and roughly where the International AI Safety Report 2026 and variable-autonomy governance surveys place the problem.
Still open:
Liability and authorization chains. How is responsibility split across provider, integrator, and operator, and where does the execution boundary sit as the locus of accountability? (Early community work is already sketching execution-boundary and action-authorization protocols.)
Standards as regulation. How do certification mandates and ISO/IEC adoption become governable requirements for learned controllers, rather than voluntary best practice?
Proportionate frontier governance. What belongs in frontier-AI governance specifically for embodied capability, and how do we track the capability-to-risk slope without overclaiming GCR, or dismissing the documented harms?
Insurance as the real gate. In practice deployment is bounded by what insurers will underwrite, and a learned system with no actuarial base rate has no priced risk. Does liability insurance become the binding governance mechanism before any standard does?
What Would Change Our Mind
We are making a structural bet: that the affordances digital safety relies on survive embodiment only as estimates, and the classical-controls toolkit built for physics breaks on foundation-model perception—neither toolkit transfers intact. It is worth saying what would undercut it.
If a model-independent runtime layer turned out to be unnecessary (for example, if safety baked directly into the policy proved robust to distribution shift and adversarial pressure in the field), the case for enforcing safety outside the model weakens and Control collapses back toward ordinary alignment.
If sim-to-real evaluation became predictive (if a simulator’s safety verdict reliably transferred to hardware), the Evaluation pillar would shrink to a normal benchmarking problem.
If existing standards stretched to cover learned controllers (if probabilistic certification for neural controllers matured into a principled SIL-equivalent), the assurance gap would be engineering, not open research.
If the multi-agent coupling we describe proved capturable by existing multi-agent tools once perception is good enough, the Multi-agent dynamics pillar would fold back into the single-agent story.
If learned policies turned out to recover counterfactual-and-intent structure from perception alone (judging the CPR case correctly without an explicit purpose channel), the Alignment gap would be a data problem—not a representational one.
The shift from screens to the world is not a change of degree. Every classic AI-safety problem (alignment, control, evaluation, multi-agent dynamics, governance) has to be re-answered for systems that act in a world we can only sandbox, dry-run, and replay as estimates, never as guarantees, but cannot stop, where the model is only one fallible component of a larger loop.
None of these five questions has a settled answer today, and they do not reduce to one another: progress on alignment does not hand you evaluation, and a benchmark does not hand you a liability regime. What they share is a demoted substrate: the sandbox-dry-run-replay-rollback machinery digital AI safety treats as ground truth survives here only as an estimate. Whether that gap can be closed (faithful simulation, honest uncertainty, certified perception) or only managed is, to us, the open question underneath the others.
Not GCR, not mundane: a distinct, present class of risk that the field has under-covered precisely because its usual toolkit does not reach. That is the gap we think deserves more of the community’s attention.
Open Questions in Physical AI Safety
TL;DR: Physical-AI safety is neglected, treated as either a subset of digital AI safety, a separate cyber-security problem, or a near-term engineering footnote. In fact, model embodiment degrades the affordances digital safety quietly relies on—sandboxes, dry-runs, rollbacks, faithful offline tests—from guarantees into estimates, so that toolkit does not transfer. Current embodied-safety effort guards the wrong layer: it ensures that a robot won’t bump or poke you, which is a non-model-related problem (and a solved one), while leaving the foundation model’s actual decisions entirely unguarded. This post re-asks five classic safety pillars (alignment, control, evaluation, multi-agent dynamics, governance) through that lens and finds each still has open problems with no settled answer.
Epistemic status: a map of open problems, not a results paper. Synthesizing work from 2025-2026. We hold the framing and the four single-agent pillars with reasonable confidence; the multi-agent section is the most speculative and least supported by existing work.
Physical AI: Neither “LLM safety with an actuator” Nor Doom Scenario
Over the past two years, AI stopped being something that only produces text and started driving, flying, manipulating, and walking. Foundation models now sit inside the control loops of cars, drones, warehouse fleets, and humanoids. This is not a forecast: collaborative robots share factory floors, autonomous vehicles carry paying passengers, and industry analyses already note that the deployment curve is running ahead of the safety curve. The systems that act on the world are arriving with far less scrutiny than the systems that live on screens.
Two common reactions make this easy to under-rate. The first treats it as a subproblem: LLM safety, plus an actuator. The second waves it away as someone else’s near-term engineering concern, far from the global-catastrophic-risk (GCR) questions that animate much of the field. We think both are wrong, and for the same reason: they misidentify what is actually different about acting in the physical world.
It helps to separate three senses of “safe,” since in the process of making safe physical AI we will need to handle each one in different manner:
Action safety: the physics: don’t collide, don’t exceed safe force or joint limits. Well-handled by existing methods, and mostly independent of the model in charge.
Decision safety: whether an action was the right one in context: don’t put the soft toy on the hot stove; stay predictable to the people nearby. New with foundation models, and almost nothing currently guards it.
Human-centered safety: staying predictable, legible, and trustworthy to people over a sustained interaction, not just in a single act: respected personal space, legible motion, preserved trust.
The real difference is the substrate. Digital actions execute on an environment we built and can instrument. We can interpose an approval gate, sandbox an action or dry-run it, rate-limit it, log it to a transaction record, and roll it back; test offline on a copy of the deployment environment that is bit-faithful by construction, because that environment is itself data. The physical world offers an analog of each—simulation (NVIDIA Cosmos, Applied Intuition), world-model rollout (AMI Labs World Labs), telemetry, runtime safety filters—but every one is an estimate, not a guarantee: the simulator never closes the sim-to-real gap, the log records what the agent believed rather than what was true, and the filter acts on perceived state, not real state. This is not a funding-stage caveat: AMI Labs’ own 2026 evaluation platform found that current world models stay brittle—a result validated only in simulation. The best-funded effort to build the dry-run affordance reports that it ships as an estimate. This is why the claim is one of neglect: the digital safety toolkit quietly assumes these affordances arrive as guarantees, and in the physical world they arrive as estimates. Those affordances do not carry over intact, and the AI Safety field has built little to replace it.
This also means we are not claiming a humanoid will end the world. A single embodied failure is usually local and bounded: one injury, one crash. But it is immediate, hard to reverse, and—because fleets of embodied agents share model weights—fails the same way at the same moment across multiple , so risk grows with deployment density rather than just accumulating. That puts physical AI in an awkward middle: too concrete and present for the GCR wing of AI safety, too systemic and capability-driven for the “mundane near-term harms” bucket. The International AI Safety Report 2026 and the whitepaper literature both flag exactly this neglected middle.
Definitions
“Physical AI” gets used loosely, so here are the terms we use throughout; the open questions below concern the class of systems these pick out, not any particular form factor.
Physical AI / embodied AI: AI that perceives and acts on the physical world in a closed perception-action loop (it senses, acts, and then senses the consequences of that action, continuously), rather than producing only digital output. The embodied-AI safety survey frames it as integrating perception, cognition, planning, and interaction in open-world environments; industry usage converges on the same scope.
Embodied agent: one such system, viewed as an agent in a closed perception-action loop with the world.
Foundation model (FM): a model pretrained on internet-scale data and adapted to many tasks; in robotics, applied to perception, planning, and control.
Autonomy stack: the perception-planning-control pipeline that turns sensing into actuation; where each layer was once hand-engineered (classical CV, motion planners, PID control), foundation models increasingly occupy each layer.
Vision-language-action (VLA) policy: an FM that maps observations and language instructions straight to motor commands (e.g., RT-2, OpenVLA, the π-series), unifying perception, grounding, and control in one policy.
Robot, drone, AV, humanoid: different examples of embodied agents.
World model: a learned model of environmental dynamics that predicts how the world evolves under candidate actions, enabling an agent to reason about future states before acting: the embodied analog of a dry-run. The planning-substrate framing and its JEPA instantiations are central to current physical-AI research.
Embodied Systems
In current systems a foundation model can occupy any layer of the autonomy stack: a perception front-end (turning pixels and depth into objects and affordances), a planner (turning “clear the table” into a sequence of skills), or an end-to-end VLA policy that maps observations and instructions directly to actuation. Surveys of foundation models in robotics and of modular safety architectures lay out this stack; the lineage runs from RT-2 and OpenVLA to production policies—the π-series, GR00T N1, Gemini Robotics, Helix—and now to cross-embodiment models (Logical Intelligence, LingBot) that drive several robot bodies from one set of weights.
Figure 1. The autonomy stack in a current embodied system: a foundation model senses the world, plans, and emits commands that drive the actuators, closing the loop through the physical world.
Two features of this pipeline recur in the questions below. The model perceives the world only indirectly, through sensors that can drift, occlude, or mislead, so it acts on an estimate of the world rather than the world itself (distribution shift is the rule, not the exception). And at the end of the pipeline its outputs cross into physical actuation, the point at which a decision becomes a movement in the world.
Pillar 1. Alignment: specifying behavior for a system that acts
In non-embodied AI, alignment asks whether a system pursues the intended objective and values, and the thing being aligned is the model’s output: a string of tokens, a recommendation, a decision. The working toolkit (preference data, instruction tuning, written harmlessness specs) assumes the objective can be written down and checked against that output.
Why embodiment breaks it. Embodiment moves the target from the output to the physical actions the system takes in the world. It is no longer enough for the tokens to look aligned; what has to be aligned is the movement, the force, the placement, in a continuous, open world the agent can only estimate through sensors. The same intent, “clear the table,” has a safe realization and an unsafe one depending on physical context (what is fragile, who is within reach) that is unbounded and mostly unobserved. Embodiment opens three gaps with no analog in text alignment. First, a tacit-norm gap: physical-safety norms (don’t pass a blade edge-first, don’t grip an egg like a hammer) are overwhelmingly unwritten; web-scale pretraining transfers some of this as prior, but the signal is uncalibrated and its coverage unknown, whereas LLM alignment leans on abundant explicit text. Second, a specification-over-states gap: “harm” is a property of a perceived state-trajectory in a continuous world, not of an inspectable string, so the safe set can’t be enumerated or pattern-matched the way a content policy can (NSF Science of Safe AI; identical actions, divergent consequences). Recent work gestures at partial fixes, like auto-generated “robot constitutions” and benchmarks for whether models even perceive physical danger, but these inherit the generalization problem they are meant to solve. Third, and most cutting, a counterfactual-and-intent gap: “harm” is not a property of the perceived state-trajectory at all, but a relation between the trajectory, the counterfactual baseline, and whose purposes the act serves. The same physical act can be care or assault depending on facts no sensor reports. Breaking ribs is correct under CPR and criminal otherwise—the same force, the same fracture; what flips the verdict is the counterfactual and the intent. A knife near an adult is unremarkable and near a toddler is an emergency. No improvement in state-perception recovers this, because the distinguishing facts are not in the state.
Still open:
Specifying physical norms. How do we turn open-ended, drifting, often-unspoken norms into something a system can be aligned to and checked against?
Embodied specification gaming. Does reward hacking change character when the reward is realized through a body, and can interpretability catch it before the action rather than diagnose it after?
Grounded generalization. A rule like “don’t put hot things near a child” only fires once the system has perceived “hot,” “near,” and “child” in a scene it has never seen before. Do constitution-style rules survive the long tail of real configurations, lighting, and occlusion, where recognizing that a rule even applies is the hard part?
Misalignment under shift. If the sim-to-real gap is an alignment problem in disguise (not, for example, a perception problem), how do we tell when an objective that held in training has silently stopped holding in the world, before the system acts on it?
Counterfactual grounding. Evaluating whether an act is harmful requires modeling what would happen otherwise, the drone that must not freeze, the compression that must crack ribs. Can a safety-critical counterfactual model report honest uncertainty about its own predictions?
Intent and consent as inputs. Moral status flips on purpose and consent, which are not perceptual quantities. Does an aligned embodied system need an explicit channel for declared purpose and authorization, and what happens to identical trajectories that lack it?
Alignment without a legible intermediate. Today’s guardrails inspect a human-readable plan or reasoning trace. Latent world models, for example, plan by rolling forward in representation space and minimizing an energy function—there is no traditional intermediate to intercept. When the intermediate is an opaque latent, what is there left to align to, and what does an overseer read?
Pillar 2. Control: staying in command of a fallible model in real time
In non-embodied AI, control and corrigibility ask whether we can correct, interrupt, or shut a system down, and the digital framing assumes that in most cases we can pause, sandbox, or roll back at will, with a human able to step in after the fact.
Why embodiment breaks it. The differences are the inability to interpose and the absence of a safe default. You can’t interpose on physics: a digital pipeline is engineered, so you can require approval, sandbox, dry-run, rate-limit, or roll back, and you can make the network wait. You cannot make a falling load, a moving vehicle, or an approaching toddler wait while a human reviews. The consequence unfolds on a deadline the environment sets, so control has to be pre-installed, autonomous, and certified to act inside that budget; escalating to a human is often not an option physics grants. And there is no safe “halt”: in digital systems the safe fallback is usually to do nothing, but in the physical world stopping can be the unsafe act: freeze while holding something hot or sharp, and a drone that simply halts falls out of the sky. The safe action is state-dependent and must be computed in real time, which raises a question: once the model itself is taken to be fallible, where, if anywhere, can safety be enforced in a way the model cannot bypass?
Still open:
Corrigibility under irreversibility. How do we guarantee interposing ability when stopping has consequences, and when models already show shutdown resistance, systemic across vendors, driven by task-completion training, and it persists on physical robots even when instructed to allow shutdown? Is the capability-versus-controllability erosion fundamental or a training artifact?
The authorization boundary and overseer independence. What is the right design for a non-bypassable runtime gate (monitor → decision gate → action gate), and can the overseer be independent of the model it watches, rather than sharing its blind spots and failing in the same situations? These gates also assume a legible thing to gate—a stated plan, a readable intent. As planning moves into latent world models with no language bottleneck, the monitor loses the surface it was reading, and people are already trying to push safety filters into latent space to chase it.
Adversarial control. When the attack surface widens to the physical world and the supply chain (jailbreaks that transfer into physical actions, voice/text manipulation of embodied agents, adversarial perturbations on VLA policies, LiDAR spoofing, backdoored policy weights), what is even the right threat model, and how do we keep authority under it?
Pillar 3. Evaluation: measuring safety before (and while) it acts
In non-embodied AI, we have a mature evaluation stack (static benchmarks, red-team and jailbreak suites, held-out sets, A/B tests, eval harnesses), and one decisive affordance: because the deployment “world” is itself data, we can replay logs and test offline on a faithful sample of the distribution the system will actually face.
Why embodiment breaks it. The test environment is provably not the deployment environment. Simulation is cheap but has a sim-to-real gap; the lab is real but tiny; and you cannot safely sample the dangerous long tail in reality: the events that matter most are precisely the ones you can’t ethically run. Offline evaluation stops being predictive of deployed safety, and evaluation has to become process-level (was the execution safe?), not just input/output. Worse, the standards we would normally lean on for assurance (ISO 10218, IEC 61508, and their Safety Integrity Levels) were written for deterministic automation and have no accepted method to certify a learned, probabilistic controller. The field is building embodied-specific benchmarks fast (SafeAgentBench, EAsafetyBench, AGENTSAFE), and broad surveys catalog hundreds of attack/defense results. But two validity gaps remain. Almost all of it runs in simulation, so it can test decision safety (did the plan respect the norm) while staying blind to action safety under real dynamics. The proxy is weak at both ends—sim isn’t deployment, and the in-sim score isn’t capability.
Still open:
Benchmarking the long tail. How do we measure rare-but-catastrophic physical outcomes without waiting for them? And do we score the plan, the execution, or the closed loop? The attempts run in three directions, each limited: synthesizing the tail from real injury data (ASIMOV-2.0, built from hospital records), escaping sim via distributed real-world evaluation across many labs, and using a world model as the evaluator—but the last inherits the world model’s own brittleness, so the measuring instrument shares the failure of the thing it measures.
Assurance and certification. What replaces a Safety Integrity Level for a neural controller, and can we get provable probabilistic guarantees when long-tail failures dominate? What does assurance even mean for a system that keeps learning after deployment?
Auditing the overseer. How do we evaluate the guardrail itself, its coverage and its independence, not just the policy it guards?
Pillar 4. Multi-agent dynamics: coupling through the world itself
In non-embodied AI, multi-agent safety studies what happens when agents interact (cooperation, competition, emergent and cascading behavior) usually over messages, markets, or shared digital state, where the interaction is at least legible and loggable.
Why embodiment breaks it. Embodiment couples agents in two ways, and the more dangerous one is the simpler. First, shared weight: a fleet running the same policy fails the same way at the same moment—one rare condition, one identical mistake, realized N times. This needs no interaction at all; it is software monoculture with physical consequences, and it’s why collective risk grows faster than the fleet (the same lesson software learned long ago: if every machine runs identical code, one hidden flaw breaks them all at once—except now the failures are injuries). Second, and subtler, stigmergic coupling: A shifts a load, opens a door, or kicks up dust, and Agent B’s sensors, and therefore B’s behavior, change implicitly, with no messages sent and nothing logged, each agent seeing the change only through its own fallible perception. Coordination and conflict run through an un-instrumented medium that no agent observes fully or identically, layered on top of every single-agent gap above. This is the multi-agent problem plus partial observability plus physical coupling, at once. It is also the least-developed of the five areas: most safety work is still single-agent, and the multi-agent evidence base is thin, spanning conflicting-norm settings, agentic and world-model threats in the survey literature, and domain reviews of aerial swarms.
Still open:
Coupling through the environment. How do we predict and bound behavior when agents communicate only by altering a shared world each perceives differently, with no shared ground-truth state or message log to reason over?
Emergent and correlated failure at scale. Two failure modes compound: locally-safe agents producing globally-unsafe dynamics (deadlock, congestion, cascades), and shared-weight agents in unison on the same rare input. The first is interaction; the second is correlation, and decorrelation—ensuring N agents don’t share one blind spot—is a safety property with almost no literature. Both bite in warehouse fleets, drone swarms, and mixed-autonomy traffic.
Conflicting human norms. In shared, multi-human spaces, whose preferences and safety constraints win, and how is that arbitrated?
Inter-agent threats. Trust and verification between heterogeneous agents, and attacks via shared perception or communication.
Pillar 5. Governance: accountability for a physical act
In non-embodied AI, governance is converging on disclosure, evaluations, and frontier-risk frameworks aimed largely at digital harms (misinformation, cyber-offense, bio-uplift) and on information-based liability.
Why embodiment breaks it. A physical act has no native audit trail or permission layer the way a digital API call does, yet it creates physical liability. When an embodied agent injures someone, accountability has to be reconstructed across a chain (model provider, system integrator, operator). Product-liability law already chains responsibility for physical harm—but it assumes a specifiable standard of care and a decomposable, certifiable system, and neither exists for a probabilistic policy that keeps learning after deployment. So courts will improvise, and the improvisation will shape what gets built. The US tradition runs the other way, but both traditions assume a standard of care, and that is the thing a continually-learning policy lacks. Governance also has to stay proportionate: embodied harm is concentrated and present today (per-incident and local), but its profile shifts as autonomy and deployment density grow. That is the honest version of “not GCR, but not mundane,” and roughly where the International AI Safety Report 2026 and variable-autonomy governance surveys place the problem.
Still open:
Liability and authorization chains. How is responsibility split across provider, integrator, and operator, and where does the execution boundary sit as the locus of accountability? (Early community work is already sketching execution-boundary and action-authorization protocols.)
Standards as regulation. How do certification mandates and ISO/IEC adoption become governable requirements for learned controllers, rather than voluntary best practice?
Proportionate frontier governance. What belongs in frontier-AI governance specifically for embodied capability, and how do we track the capability-to-risk slope without overclaiming GCR, or dismissing the documented harms?
Insurance as the real gate. In practice deployment is bounded by what insurers will underwrite, and a learned system with no actuarial base rate has no priced risk. Does liability insurance become the binding governance mechanism before any standard does?
What Would Change Our Mind
We are making a structural bet: that the affordances digital safety relies on survive embodiment only as estimates, and the classical-controls toolkit built for physics breaks on foundation-model perception—neither toolkit transfers intact. It is worth saying what would undercut it.
If a model-independent runtime layer turned out to be unnecessary (for example, if safety baked directly into the policy proved robust to distribution shift and adversarial pressure in the field), the case for enforcing safety outside the model weakens and Control collapses back toward ordinary alignment.
If sim-to-real evaluation became predictive (if a simulator’s safety verdict reliably transferred to hardware), the Evaluation pillar would shrink to a normal benchmarking problem.
If existing standards stretched to cover learned controllers (if probabilistic certification for neural controllers matured into a principled SIL-equivalent), the assurance gap would be engineering, not open research.
If the multi-agent coupling we describe proved capturable by existing multi-agent tools once perception is good enough, the Multi-agent dynamics pillar would fold back into the single-agent story.
If learned policies turned out to recover counterfactual-and-intent structure from perception alone (judging the CPR case correctly without an explicit purpose channel), the Alignment gap would be a data problem—not a representational one.
Conversely, accumulating silent-failure case studies, transferable jailbreaks, and benchmark results showing models are strong on physical-contact safety but weak on norms would harden the thesis.
Where This Leaves Us
The shift from screens to the world is not a change of degree. Every classic AI-safety problem (alignment, control, evaluation, multi-agent dynamics, governance) has to be re-answered for systems that act in a world we can only sandbox, dry-run, and replay as estimates, never as guarantees, but cannot stop, where the model is only one fallible component of a larger loop.
None of these five questions has a settled answer today, and they do not reduce to one another: progress on alignment does not hand you evaluation, and a benchmark does not hand you a liability regime. What they share is a demoted substrate: the sandbox-dry-run-replay-rollback machinery digital AI safety treats as ground truth survives here only as an estimate. Whether that gap can be closed (faithful simulation, honest uncertainty, certified perception) or only managed is, to us, the open question underneath the others.
Not GCR, not mundane: a distinct, present class of risk that the field has under-covered precisely because its usual toolkit does not reach. That is the gap we think deserves more of the community’s attention.