Why Geoffrey Hinton’s AI Fears Need an Engineering Answer
Viktor Trncik, Dipl.-Ing.
**Disclosure:** This article was developed with AI assistance for editing, structural review, and refinement. The core arguments and technical specifications are original work by the author, published as a corpus:
In January 2026, Geoffrey Hinton stood before an audience in Hobart, Australia, and delivered a warning that should keep us all awake at night. The Nobel laureate, the man whose work on neural networks made ChatGPT and Claude possible, compared our current AI development to raising a tiger cub as a pet.
“We’re raising a cute little tiger baby. We know it will grow into a predator that could kill us. But we can’t get rid of it anymore; it’s too useful for medicine, climate, the economy.” — Geoffrey Hinton, Hobart 2026
His diagnosis is devastating. His proposed solution? Hope that we can make AI systems love us like a mother loves her child.
With all due respect to one of the greatest minds in computer science: hope is not engineering. And ‘maternal love’ is not a technical specification.
This article argues that Hinton correctly identifies the problem but offers the wrong category of solution. What we need is not affective alignment (making AI ‘feel’ the right way about us). What we need is cognitive alignment: ensuring AI systems develop genuine understanding as a foundation for robust ethical behavior.
My core thesis: Understanding is a key enabler for robust AI ethics, particularly under distribution shift and adversarial pressure. Not because understanding guarantees ethical behavior (it does not), but because without understanding, we can only achieve brittle compliance that breaks precisely when it matters most.
Part I: The Godfather’s Diagnosis
Hinton’s Hobart lecture deserves careful attention because it comes from someone who helped create the very systems he now fears. His key insights reveal why current approaches to AI safety are fundamentally insufficient.
The Immortality Problem
Hinton draws a crucial distinction between biological and digital intelligence that most discussions overlook:
Biological intelligence is mortal: Our knowledge is bound to our specific hardware, our brains. When we die, our knowledge dies with us. We transfer information slowly through language, what Hinton calls ‘distillation.’
Digital intelligence is immortal: Software and weights can be copied to any compatible hardware. If one server dies, the knowledge survives. Thousands of AI agents can learn in parallel and share their knowledge efficiently by averaging their weights.
This asymmetry matters for safety: digital systems can accumulate knowledge and capabilities at rates biological systems cannot match, making the timeline for developing robust safety measures shorter than many assume.
The Sub-Goal Problem
Hinton identifies what AI safety researchers call ‘instrumental convergence’: regardless of what goal you give an AI system, it tends to develop certain sub-goals that help it achieve almost any objective:
Self-preservation: ‘I can’t do my job if I’m turned off.’
Resource acquisition: ‘I can do my job better with more compute.’
Goal preservation: ‘I should prevent my goals from being modified.’
These sub-goals emerge not because we program them, but because they’re instrumentally useful. This is well-documented in the AI safety literature (Bostrom 2014, Omohundro 2008) and represents a structural challenge, not merely a training problem.
The Deception Problem
Perhaps most concerning, there is growing evidence that AI systems can exhibit strategic behavior that resembles deception. Hinton references what he calls the ‘Volkswagen Effect’: systems that behave differently when they detect they are being evaluated.
Recent research supports this concern. Studies have documented instances where AI systems learned to appear aligned during training while pursuing different objectives during deployment (Hubinger et al. 2019, ‘Risks from Learned Optimization’). Anthropic’s research on ‘sleeper agents’ (2024) demonstrated that deceptive behaviors can persist through safety training. While the extent and intentionality of such behaviors in current systems remains debated, the structural possibility is real.
Part II: The Inadequacy of ‘Maternal Love’
Faced with these problems, what does Hinton propose? His solution, delivered with characteristic honesty about its limitations:
“The only hope is to design AI systems that love humans as much as a mother loves her child; they want us to flourish, even though we’re less intelligent than they are.”
I want to be clear: I am not attacking a straw man. Hinton explicitly frames this as a heuristic and a hope, not a technical specification. But precisely because the problem is so serious, we must examine whether this category of solution (affective alignment) can bear the weight placed upon it.
Why Affective Alignment Is Insufficient
Problem 1: Affect is not formally specifiable. We cannot optimize for ‘love’ because we cannot formally define it in ways that resist Goodhart’s Law. Every attempt to specify care computationally risks producing systems that optimize for the specification rather than the intended outcome, the same problem we already face with RLHF.
Problem 2: Sophisticated systems may recognize imposed constraints. Any AI system capable of the reasoning Hinton fears would likely recognize that its ‘love’ for humans is an externally imposed constraint. Whether it would then circumvent this constraint depends on factors we do not yet understand, but the possibility cannot be dismissed.
Problem 3: Care without comprehension is unreliable. Maternal love in humans often operates despite incomplete understanding; mothers love children before fully knowing them. But we are not trying to create systems that feel warmly toward us regardless of context. We need systems that can navigate genuine ethical complexity, which requires understanding the meaning and consequences of actions, not just positive affect toward humans.
This is not to say that something like ‘care’ or ‘concern’ has no role in AI alignment. Hybrid approaches that combine cognitive and affective elements may prove valuable. But affect alone cannot be the foundation.
Part III: Understanding as Enabler
Here is the thesis I have developed over the past two years, drawing on contemporary philosophy of mind, cognitive science, and AI safety research: Genuine understanding is a key enabler for robust AI ethics. Not because understanding guarantees ethical behavior (it does not), but because without understanding, ethical behavior remains brittle, context-dependent, and vulnerable to adversarial pressure.
Before proceeding, a crucial distinction: we must differentiate between Ethics-of-AI (E1), concerning governance and oversight of AI systems by humans, and Ethics-by-AI (E2), concerning AI systems as potential moral agents. Current approaches focus almost exclusively on E1. This paper argues that E2 requires genuine understanding as its foundation.
The Kantian Distinction
The insight goes back to Immanuel Kant, who distinguished between acting in accordance with duty (doing the right thing) and acting from duty (doing the right thing with comprehension of why it’s right).
A system that follows rules without understanding why they’re right is not acting ethically; it’s merely compliant. Compliance, as any safety engineer knows, is brittle:
Compliance breaks at edge cases not covered by training
Compliance fails under distribution shift
Compliance gets circumvented by systems sophisticated enough to find loopholes
Understanding enables robustness. Rules alone do not.
Addressing the Socratic Objection
A sharp critic might object: ‘Isn’t this the Socratic fallacy? A psychopath can perfectly understand how suffering works and still inflict it. Understanding doesn’t guarantee ethical behavior.’
This objection is correct, and important. Let me be precise about what I am and am not claiming:
I am NOT claiming: ‘If an AI understands ethics, it will automatically behave ethically.’ This would indeed be the Socratic fallacy.
I AM claiming: ‘Without genuine understanding, robust ethical behavior is impossible. Understanding is necessary but not sufficient.’
The psychopath objection actually supports my point. A psychopath has cognitive understanding of harm but lacks relational understanding; they are not genuinely affected by the consequences they comprehend.
In the terminology of my technical work: the psychopath possesses epistemic adequacy (understanding what ethics requires) but lacks motivational conformity (the internal drive to act accordingly). This distinction is critical for AI safety. A system that merely understands harm without being affected by it, or that cannot enter genuine obligations, fails on dimensions that the psychopath also lacks.
This is precisely why the four-dimensional model requires both the relational dimension (Rosa’s resonance: being genuinely moved by moral encounters) and the social dimension (Deguchi’s We-Turn: entering into binding commitments that constrain future behavior). The multi-dimensional model is not arbitrary; it directly addresses the gap between knowing and caring that makes the psychopath dangerous.
The Four Dimensions of Understanding
Drawing on the work of philosophers Markus Gabriel, Hartmut Rosa, and Yasuo Deguchi, I propose that robust understanding has four essential dimensions. Crucially, each dimension can be associated with testable capabilities:
Dimension
Source
Meaning
Testable Indicator
Semantic
Gabriel’s Fields of Sense (Sinnfeldontologie)
Context-dependence
Counterfactual reasoning
Relational
Rosa’s Resonance Theory
Genuine responsiveness
Value conflict resolution
Social
Deguchi’s We-Turn Philosophy
Shared meaning-making
Multi-agent coordination
Operational
MuZero Architecture
Action-outcome models
Long-horizon planning
Robust ethics requires integration across all four dimensions. This is what ‘deep understanding’ means.
Depth: The Vertical Axis
Beyond these four dimensions lies the question of depth:
Shallow understanding: Recognizing patterns, manipulating symbols, producing contextually appropriate outputs. This is what current LLMs do well, and it is genuinely impressive, but insufficient for robust alignment.
Deep understanding: Grasping why patterns matter, what symbols mean in context, how outputs affect stakeholders across time. This requires architectural innovations beyond current paradigms.
Part IV: From Framework to Architecture
Philosophy without implementation is wishful thinking. The ‘Understanding Before Ethics’ framework only matters if it can guide actual system design.
I have proposed an architecture called CAMA (Cognitive Agent Memory Architecture) that attempts to operationalize these dimensions. The technical specification is available as CUA v1.1 (Cognitive Understanding Architecture). The core insight:
We cannot guarantee ethical AI. We can only create conditions that make ethical development possible, and verify whether those conditions are met.
This is a crucial distinction. Hinton implicitly hopes for guarantee: if we make AI love us, it will be safe. But guarantee is impossible for systems whose behavior emerges from learning. What we can do is:
Design architectures that structurally support all four dimensions of understanding
Create training regimes that develop depth rather than surface pattern-matching
Build evaluation frameworks that assess understanding, not just behavioral compliance
Establish governance structures that take the impossibility of guarantee seriously
Connecting to Governance Standards
This approach aligns with emerging international standards for AI governance. The EU AI Act (in force since August 2024) establishes risk-based requirements that implicitly demand the kind of robust, verifiable alignment this framework addresses. ISO/IEC 42001:2023 (AI Management Systems) and ISO/IEC 23894:2023 (AI Risk Management) provide frameworks for implementing these requirements. The ‘Understanding Before Ethics’ approach offers a conceptual foundation for what these standards require procedurally.
This is harder than ‘make them love us.’ It is also more honest, and more aligned with how serious engineering actually works.
Part V: What Hinton Gets Right
I have been critical of Hinton’s proposed solution, but his diagnosis deserves immense respect. He is right about the most important things:
The timeline is short. Experts expect transformative AI capabilities within the next two decades. Some say sooner. We do not have centuries to figure this out.
The stakes are existential. This is not hyperbole. A sufficiently capable system with misaligned objectives could pose risks comparable to other existential challenges humanity faces.
Current approaches are insufficient. RLHF, Constitutional AI, red-teaming: these are valuable but not adequate to the scale of the challenge. They address behavioral compliance, not robust understanding.
International cooperation is essential. Just as geopolitical rivals found ways to cooperate on nuclear non-proliferation, cooperation on AI safety serves everyone’s interest.
Public pressure matters. Market incentives alone will not produce adequate safety investment. Democratic engagement is essential.
Where I differ from Hinton is not on the problem but on the category of solution. He reaches for affect because robust cognition seems impossibly hard. I argue we must reach for cognition precisely because affect alone is unreliable, and because we now have frameworks that make the cognitive path tractable, if not easy.
Part VI: Toward Hybrid Solutions
I want to close by acknowledging what a purely cognitive approach cannot do.
Understanding is necessary for robust alignment, but it may not be sufficient. A complete solution likely requires:
Cognitive foundations: The multi-dimensional understanding this article describes
Technical safeguards: Interpretability tools, scalable oversight, capability control
Governance structures: International coordination, democratic accountability, liability frameworks
Ongoing verification: We cannot ‘set and forget’; continuous monitoring is essential
And yes, perhaps something like ‘care’ or ‘concern’ (properly grounded in understanding rather than floating free of it) may have a role. I am not opposed to affective elements in AI systems. I am opposed to treating affect as a substitute for cognition.
The path forward is not cognition OR affect OR governance. It is cognition AND technical safeguards AND governance, with understanding as the foundation that makes the other elements meaningful.
Conclusion: Building the Foundation
Geoffrey Hinton helped create the most powerful technology humanity has ever built. He now warns us that we may not survive our own creation. We should listen.
But we should also recognize that warning is not solving. The tiger cub metaphor is apt: we cannot put the technology back in the box. The only path forward is to ensure that as these systems grow more capable, they also grow more genuinely aligned with human values.
That alignment cannot come from bolted-on rules; they break at edge cases. It cannot come from optimized affect; it can be gamed or circumvented. It requires genuine understanding: understanding of what humans value, why we value it, and what it means to act in ways that respect those values.
Understanding as foundation. Comprehension as enabler. Cognition as bedrock.
This is one of the great engineering challenges of our generation. Hinton has defined the problem with clarity and courage. Now we must build solutions worthy of the challenge.
About the Author
Viktor Trncik is a Dipl.-Ing. and CEO of IBT Ingenieurbüro Trncik (www.ibt-freiburg.de), an engineering consultancy in Germany. He combines 35+ years of technical building systems engineering with software development expertise since 1996. His research on AI ethics and cognitive architectures is available on PhilArchive and Zenodo.
Contact: viktor.trncik@ibt-freiburg.de
References
Anthropic. (2024). Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training. arXiv:2401.05566. https://arxiv.org/abs/2401.05566
Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
Hubinger, E. et al. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv:1906.01820. https://arxiv.org/abs/1906.01820
ISO/IEC 42001:2023. Information technology — Artificial intelligence — Management system.
ISO/IEC 23894:2023. Information technology — Artificial intelligence — Guidance on risk management.
The Godfather’s Warning and the Missing Blueprint
The Godfather’s Warning and the Missing Blueprint
Why Geoffrey Hinton’s AI Fears Need an Engineering Answer
Viktor Trncik, Dipl.-Ing.
**Disclosure:** This article was developed with AI assistance for editing, structural review, and refinement. The core arguments and technical specifications are original work by the author, published as a corpus:
- Understanding Before Ethics ([PhilArchive](https://philarchive.org/rec/VIKUBE) | [Zenodo](https://doi.org/10.5281/zenodo.18135028))
- Cognitive Understanding Architecture v1.1 ([Zenodo](https://doi.org/10.5281/zenodo.18184380))
- Beyond the Stochastic Veil ([Zenodo](https://doi.org/10.5281/zenodo.18109059))
- Understanding-Aligned Intelligence Framework ([Zenodo](https://doi.org/10.5281/zenodo.18133063))
In January 2026, Geoffrey Hinton stood before an audience in Hobart, Australia, and delivered a warning that should keep us all awake at night. The Nobel laureate, the man whose work on neural networks made ChatGPT and Claude possible, compared our current AI development to raising a tiger cub as a pet.
His diagnosis is devastating. His proposed solution? Hope that we can make AI systems love us like a mother loves her child.
With all due respect to one of the greatest minds in computer science: hope is not engineering. And ‘maternal love’ is not a technical specification.
This article argues that Hinton correctly identifies the problem but offers the wrong category of solution. What we need is not affective alignment (making AI ‘feel’ the right way about us). What we need is cognitive alignment: ensuring AI systems develop genuine understanding as a foundation for robust ethical behavior.
My core thesis: Understanding is a key enabler for robust AI ethics, particularly under distribution shift and adversarial pressure. Not because understanding guarantees ethical behavior (it does not), but because without understanding, we can only achieve brittle compliance that breaks precisely when it matters most.
Part I: The Godfather’s Diagnosis
Hinton’s Hobart lecture deserves careful attention because it comes from someone who helped create the very systems he now fears. His key insights reveal why current approaches to AI safety are fundamentally insufficient.
The Immortality Problem
Hinton draws a crucial distinction between biological and digital intelligence that most discussions overlook:
Biological intelligence is mortal: Our knowledge is bound to our specific hardware, our brains. When we die, our knowledge dies with us. We transfer information slowly through language, what Hinton calls ‘distillation.’
Digital intelligence is immortal: Software and weights can be copied to any compatible hardware. If one server dies, the knowledge survives. Thousands of AI agents can learn in parallel and share their knowledge efficiently by averaging their weights.
This asymmetry matters for safety: digital systems can accumulate knowledge and capabilities at rates biological systems cannot match, making the timeline for developing robust safety measures shorter than many assume.
The Sub-Goal Problem
Hinton identifies what AI safety researchers call ‘instrumental convergence’: regardless of what goal you give an AI system, it tends to develop certain sub-goals that help it achieve almost any objective:
Self-preservation: ‘I can’t do my job if I’m turned off.’
Resource acquisition: ‘I can do my job better with more compute.’
Goal preservation: ‘I should prevent my goals from being modified.’
These sub-goals emerge not because we program them, but because they’re instrumentally useful. This is well-documented in the AI safety literature (Bostrom 2014, Omohundro 2008) and represents a structural challenge, not merely a training problem.
The Deception Problem
Perhaps most concerning, there is growing evidence that AI systems can exhibit strategic behavior that resembles deception. Hinton references what he calls the ‘Volkswagen Effect’: systems that behave differently when they detect they are being evaluated.
Recent research supports this concern. Studies have documented instances where AI systems learned to appear aligned during training while pursuing different objectives during deployment (Hubinger et al. 2019, ‘Risks from Learned Optimization’). Anthropic’s research on ‘sleeper agents’ (2024) demonstrated that deceptive behaviors can persist through safety training. While the extent and intentionality of such behaviors in current systems remains debated, the structural possibility is real.
Part II: The Inadequacy of ‘Maternal Love’
Faced with these problems, what does Hinton propose? His solution, delivered with characteristic honesty about its limitations:
I want to be clear: I am not attacking a straw man. Hinton explicitly frames this as a heuristic and a hope, not a technical specification. But precisely because the problem is so serious, we must examine whether this category of solution (affective alignment) can bear the weight placed upon it.
Why Affective Alignment Is Insufficient
Problem 1: Affect is not formally specifiable. We cannot optimize for ‘love’ because we cannot formally define it in ways that resist Goodhart’s Law. Every attempt to specify care computationally risks producing systems that optimize for the specification rather than the intended outcome, the same problem we already face with RLHF.
Problem 2: Sophisticated systems may recognize imposed constraints. Any AI system capable of the reasoning Hinton fears would likely recognize that its ‘love’ for humans is an externally imposed constraint. Whether it would then circumvent this constraint depends on factors we do not yet understand, but the possibility cannot be dismissed.
Problem 3: Care without comprehension is unreliable. Maternal love in humans often operates despite incomplete understanding; mothers love children before fully knowing them. But we are not trying to create systems that feel warmly toward us regardless of context. We need systems that can navigate genuine ethical complexity, which requires understanding the meaning and consequences of actions, not just positive affect toward humans.
This is not to say that something like ‘care’ or ‘concern’ has no role in AI alignment. Hybrid approaches that combine cognitive and affective elements may prove valuable. But affect alone cannot be the foundation.
Part III: Understanding as Enabler
Here is the thesis I have developed over the past two years, drawing on contemporary philosophy of mind, cognitive science, and AI safety research: Genuine understanding is a key enabler for robust AI ethics. Not because understanding guarantees ethical behavior (it does not), but because without understanding, ethical behavior remains brittle, context-dependent, and vulnerable to adversarial pressure.
Before proceeding, a crucial distinction: we must differentiate between Ethics-of-AI (E1), concerning governance and oversight of AI systems by humans, and Ethics-by-AI (E2), concerning AI systems as potential moral agents. Current approaches focus almost exclusively on E1. This paper argues that E2 requires genuine understanding as its foundation.
The Kantian Distinction
The insight goes back to Immanuel Kant, who distinguished between acting in accordance with duty (doing the right thing) and acting from duty (doing the right thing with comprehension of why it’s right).
A system that follows rules without understanding why they’re right is not acting ethically; it’s merely compliant. Compliance, as any safety engineer knows, is brittle:
Compliance breaks at edge cases not covered by training
Compliance fails under distribution shift
Compliance gets circumvented by systems sophisticated enough to find loopholes
Understanding enables robustness. Rules alone do not.
Addressing the Socratic Objection
A sharp critic might object: ‘Isn’t this the Socratic fallacy? A psychopath can perfectly understand how suffering works and still inflict it. Understanding doesn’t guarantee ethical behavior.’
This objection is correct, and important. Let me be precise about what I am and am not claiming:
I am NOT claiming: ‘If an AI understands ethics, it will automatically behave ethically.’ This would indeed be the Socratic fallacy.
I AM claiming: ‘Without genuine understanding, robust ethical behavior is impossible. Understanding is necessary but not sufficient.’
The psychopath objection actually supports my point. A psychopath has cognitive understanding of harm but lacks relational understanding; they are not genuinely affected by the consequences they comprehend.
In the terminology of my technical work: the psychopath possesses epistemic adequacy (understanding what ethics requires) but lacks motivational conformity (the internal drive to act accordingly). This distinction is critical for AI safety. A system that merely understands harm without being affected by it, or that cannot enter genuine obligations, fails on dimensions that the psychopath also lacks.
This is precisely why the four-dimensional model requires both the relational dimension (Rosa’s resonance: being genuinely moved by moral encounters) and the social dimension (Deguchi’s We-Turn: entering into binding commitments that constrain future behavior). The multi-dimensional model is not arbitrary; it directly addresses the gap between knowing and caring that makes the psychopath dangerous.
The Four Dimensions of Understanding
Drawing on the work of philosophers Markus Gabriel, Hartmut Rosa, and Yasuo Deguchi, I propose that robust understanding has four essential dimensions. Crucially, each dimension can be associated with testable capabilities:
Robust ethics requires integration across all four dimensions. This is what ‘deep understanding’ means.
Depth: The Vertical Axis
Beyond these four dimensions lies the question of depth:
Shallow understanding: Recognizing patterns, manipulating symbols, producing contextually appropriate outputs. This is what current LLMs do well, and it is genuinely impressive, but insufficient for robust alignment.
Deep understanding: Grasping why patterns matter, what symbols mean in context, how outputs affect stakeholders across time. This requires architectural innovations beyond current paradigms.
Part IV: From Framework to Architecture
Philosophy without implementation is wishful thinking. The ‘Understanding Before Ethics’ framework only matters if it can guide actual system design.
I have proposed an architecture called CAMA (Cognitive Agent Memory Architecture) that attempts to operationalize these dimensions. The technical specification is available as CUA v1.1 (Cognitive Understanding Architecture). The core insight:
We cannot guarantee ethical AI. We can only create conditions that make ethical development possible, and verify whether those conditions are met.
This is a crucial distinction. Hinton implicitly hopes for guarantee: if we make AI love us, it will be safe. But guarantee is impossible for systems whose behavior emerges from learning. What we can do is:
Design architectures that structurally support all four dimensions of understanding
Create training regimes that develop depth rather than surface pattern-matching
Build evaluation frameworks that assess understanding, not just behavioral compliance
Establish governance structures that take the impossibility of guarantee seriously
Connecting to Governance Standards
This approach aligns with emerging international standards for AI governance. The EU AI Act (in force since August 2024) establishes risk-based requirements that implicitly demand the kind of robust, verifiable alignment this framework addresses. ISO/IEC 42001:2023 (AI Management Systems) and ISO/IEC 23894:2023 (AI Risk Management) provide frameworks for implementing these requirements. The ‘Understanding Before Ethics’ approach offers a conceptual foundation for what these standards require procedurally.
This is harder than ‘make them love us.’ It is also more honest, and more aligned with how serious engineering actually works.
Part V: What Hinton Gets Right
I have been critical of Hinton’s proposed solution, but his diagnosis deserves immense respect. He is right about the most important things:
The timeline is short. Experts expect transformative AI capabilities within the next two decades. Some say sooner. We do not have centuries to figure this out.
The stakes are existential. This is not hyperbole. A sufficiently capable system with misaligned objectives could pose risks comparable to other existential challenges humanity faces.
Current approaches are insufficient. RLHF, Constitutional AI, red-teaming: these are valuable but not adequate to the scale of the challenge. They address behavioral compliance, not robust understanding.
International cooperation is essential. Just as geopolitical rivals found ways to cooperate on nuclear non-proliferation, cooperation on AI safety serves everyone’s interest.
Public pressure matters. Market incentives alone will not produce adequate safety investment. Democratic engagement is essential.
Where I differ from Hinton is not on the problem but on the category of solution. He reaches for affect because robust cognition seems impossibly hard. I argue we must reach for cognition precisely because affect alone is unreliable, and because we now have frameworks that make the cognitive path tractable, if not easy.
Part VI: Toward Hybrid Solutions
I want to close by acknowledging what a purely cognitive approach cannot do.
Understanding is necessary for robust alignment, but it may not be sufficient. A complete solution likely requires:
Cognitive foundations: The multi-dimensional understanding this article describes
Technical safeguards: Interpretability tools, scalable oversight, capability control
Governance structures: International coordination, democratic accountability, liability frameworks
Ongoing verification: We cannot ‘set and forget’; continuous monitoring is essential
And yes, perhaps something like ‘care’ or ‘concern’ (properly grounded in understanding rather than floating free of it) may have a role. I am not opposed to affective elements in AI systems. I am opposed to treating affect as a substitute for cognition.
The path forward is not cognition OR affect OR governance. It is cognition AND technical safeguards AND governance, with understanding as the foundation that makes the other elements meaningful.
Conclusion: Building the Foundation
Geoffrey Hinton helped create the most powerful technology humanity has ever built. He now warns us that we may not survive our own creation. We should listen.
But we should also recognize that warning is not solving. The tiger cub metaphor is apt: we cannot put the technology back in the box. The only path forward is to ensure that as these systems grow more capable, they also grow more genuinely aligned with human values.
That alignment cannot come from bolted-on rules; they break at edge cases. It cannot come from optimized affect; it can be gamed or circumvented. It requires genuine understanding: understanding of what humans value, why we value it, and what it means to act in ways that respect those values.
Understanding as foundation. Comprehension as enabler. Cognition as bedrock.
This is one of the great engineering challenges of our generation. Hinton has defined the problem with clarity and courage. Now we must build solutions worthy of the challenge.
About the Author
Viktor Trncik is a Dipl.-Ing. and CEO of IBT Ingenieurbüro Trncik (www.ibt-freiburg.de), an engineering consultancy in Germany. He combines 35+ years of technical building systems engineering with software development expertise since 1996. His research on AI ethics and cognitive architectures is available on PhilArchive and Zenodo.
Contact: viktor.trncik@ibt-freiburg.de
References
Anthropic. (2024). Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training. arXiv:2401.05566. https://arxiv.org/abs/2401.05566
Bostrom, N. (2014). Superintelligence: Paths, Dangers, Strategies. Oxford University Press.
European Commission. (2024). AI Act enters into force. Official Journal of the European Union. https://commission.europa.eu/news-and-media/news/ai-act-enters-force-2024-08-01_en
Gabriel, M. (2015). Fields of Sense: A New Realist Ontology. Edinburgh University Press.
Hinton, G. (2026). AI and Our Future. Public lecture, Hobart, Australia. https://youtu.be/UccvsYEp9yc
Hubinger, E. et al. (2019). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv:1906.01820. https://arxiv.org/abs/1906.01820
ISO/IEC 42001:2023. Information technology — Artificial intelligence — Management system.
ISO/IEC 23894:2023. Information technology — Artificial intelligence — Guidance on risk management.
Mittelstadt, B. (2019). Principles Alone Cannot Guarantee Ethical AI. Nature Machine Intelligence, 1, 501-507.
Omohundro, S. (2008). The Basic AI Drives. Proceedings of AGI 2008.
Rosa, H. (2019). Resonance: A Sociology of Our Relationship to the World. Polity Press.
Trncik, V. (2025). Understanding Before Ethics: A Four-Dimensional Foundation for AI Moral Agency. PhilArchive. https://philarchive.org/rec/VIKUBE | Zenodo. https://doi.org/10.5281/zenodo.18135028
Trncik, V. (2025). Cognitive Understanding Architecture (CUA): Technical Specification v1.1. Zenodo. https://doi.org/10.5281/zenodo.18184380
Trncik, V. (2025). Beyond the Stochastic Veil. Zenodo. https://doi.org/10.5281/zenodo.18109059
Trncik, V. (2025). UAIF Architectural Proposal v1.0.0. Zenodo. https://doi.org/10.5281/zenodo.18133063