Biosurveillance (e.g. metagenomic sequencing networks, atmospheric and environmental sensors for CBRN)
Hardening
Formal verification of critical infrastructure (energy grids, defense weapons, healthcare systems). Includes simply rewriting vintage software into more secure programming languages àla The Great Refactor (in Rust). Can focus on microkernels, parsers, safety controllers, update managers, and interlock logic.
Analog/mechanical backups to critical infrastructure so that even total digital collapse doesn’t cause civilizational collapse
Quantum computing cybersecurity defense
Electromagnetic pulse (EMP) hardening of critical electronics
Decentralized food and energy production
Distributed solar + battery + vertical farming systems that reduce the fragility of centralized supply chains. If an ASI attack targets grid infrastructure or food logistics, distributed systems would degrade more gracefully.
Physical interlocks for air-gapped critical infrastructure
Screening is done at the synthesis-level and cryptographically-proven that it ran through screening and passed (this is largely voluntary as of now). Basically, a machine would not be able to synthesize a flagged sequence.
Response
Autonomous rapid-response manufacturing
Facilities that can pivot from general-purpose to producing vaccines, antivirals, or PPE within days of a threat detection. DARPA’s P3 program explored this; the atoms-world bottleneck is having the physical plant pre-positioned and the regulatory frameworks pre-approved.
Broad-spectrum antivirals and pan-coronavirus/pan-influenza prophylactics
Physical kill switches for data centers
Whole-brain emulation tech
Cloud lab infrastructure with significant safety guarantees
Some of these measures run the risk of empowering malicious AI to gain greater control over human use of technology. Chip-level “safety” mechanisms could enable AI surveillance of human technology useas well as surveillance of AI by humans. EMP hardening can protect AI from human defensive action.
In a conflict between incipient malicious AI and humanity-in-general, I fear that some human authorities will be on the side of AI. Accelerationism and even successionism are active in the halls of power today. Any measure that has to be entrusted to specific authorities runs the risk of those authorities being subverted … or voluntarily handing over the keys to the kingdom.
Which technologies in the world of atoms would make us safer from ASI?
I’m trying to write down a portfolio of defensive technologies touching the physical world.
I’m thinking about this largely because I’m trying to identify things I could accelerate in my startup.
Here are some initial ideas:
Detection
Chip-level safety mechanisms (e.g. on-chip location attestation, tamper-evident supply chain for chips)
Biosurveillance (e.g. metagenomic sequencing networks, atmospheric and environmental sensors for CBRN)
Hardening
Formal verification of critical infrastructure (energy grids, defense weapons, healthcare systems). Includes simply rewriting vintage software into more secure programming languages àla The Great Refactor (in Rust). Can focus on microkernels, parsers, safety controllers, update managers, and interlock logic.
Analog/mechanical backups to critical infrastructure so that even total digital collapse doesn’t cause civilizational collapse
Quantum computing cybersecurity defense
Electromagnetic pulse (EMP) hardening of critical electronics
Decentralized food and energy production
Distributed solar + battery + vertical farming systems that reduce the fragility of centralized supply chains. If an ASI attack targets grid infrastructure or food logistics, distributed systems would degrade more gracefully.
Physical interlocks for air-gapped critical infrastructure
Indoor-air biosecurity (germicidal UV, HVAC integration)
Pharmaceutical supply chain integrity
DNA synthesis screening hardware
Screening is done at the synthesis-level and cryptographically-proven that it ran through screening and passed (this is largely voluntary as of now). Basically, a machine would not be able to synthesize a flagged sequence.
Response
Autonomous rapid-response manufacturing
Facilities that can pivot from general-purpose to producing vaccines, antivirals, or PPE within days of a threat detection. DARPA’s P3 program explored this; the atoms-world bottleneck is having the physical plant pre-positioned and the regulatory frameworks pre-approved.
Broad-spectrum antivirals and pan-coronavirus/pan-influenza prophylactics
Physical kill switches for data centers
Whole-brain emulation tech
Cloud lab infrastructure with significant safety guarantees
Some of these measures run the risk of empowering malicious AI to gain greater control over human use of technology. Chip-level “safety” mechanisms could enable AI surveillance of human technology use as well as surveillance of AI by humans. EMP hardening can protect AI from human defensive action.
In a conflict between incipient malicious AI and humanity-in-general, I fear that some human authorities will be on the side of AI. Accelerationism and even successionism are active in the halls of power today. Any measure that has to be entrusted to specific authorities runs the risk of those authorities being subverted … or voluntarily handing over the keys to the kingdom.