The only complete and comprehensive solution that can make AIs 100% safe: in a nutshell we need to at least lobby politicians to make GPU manufacturers (NVIDIA and others) to make robust blacklists (whitelists and new non-agentic hardware, please, read on) of bad AI models, update GPU firmwares with them. It’s not the full solution: please steelman and read the rest to learn how to make it much safer and why it will work (NVIDIA and other GPU makers will want to do it because it’ll double their business and all future cash flows. Gov will want it because it removes all AI threats from China, all hackers, terrorists and rogue states):
The elephant in the room: even if current major AI companies will align their AIs, there will be hackers (can create viruses with agentic AI component to steal money), rogue states (can decide to use AI agents to spread propaganda and to spy) and military (AI agents in drones and to hack infrastructure). So we need to align the world, not just the models:
Imagine a agentic AI botnet starts to spread on user computers and GPUs, GPUs are like nukes to be taken, they are not protected from running bad AI models at all. I call it the agentic explosion, it’s probably going to happen before the “intelligence-agency” explosion (intelligence on its own cannot explode, an LLM is a static geometric shape—a bunch of vectors—without GPUs). Right now we are hopelessly unprepared. We won’t have time to create “agentic AI antiviruses”.
To force GPU and OS providers to update their firmware and software to at least have robust updatable blacklists of bad AI (agentic) models. And to have robust whitelists, in case there will be so many unaligned models, blacklists will become useless.
We can force NVIDIA to replace agentic GPUs with non-agentic ones. Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output. They probably shouldn’t connect to the Internet, use tools, or we should be able to limit that in case we’ll need to.
This way NVIDIA will have the skin in the game and be directly responsible for the safety of AI models that run on its GPUs.
The same way Apple feels responsible for the App Store and the apps in it, doesn’t let viruses happen.
NVIDIA will want it because it can potentially like App Store take 15-30% cut from OpenAI and other commercial models, while free models will remain free (like the free apps in the App Store).
Replacing GPUs can double NVIDIA’s business, so they can even lobby themselves to have those things. All companies and CEOs want money, have obligations to shareholders to increase company’s market capitalization. We must make AI safety something that is profitable. Those companies that don’t promote AI safety should go bankrupt or be outlawed.
It seems extremely difficult to make a blacklist of models in a way that isn’t trivially breakable. (E.g. what’s supposed to happen when someone adds a tiny amount of noise to the weights of a blacklisted model, or rotates them along a gauge invariance?)
Yes, Buck, thank you for responding! A robust whitelist (especially hardware level, each GPU can become a computer for securing itself) potentially solves it (of course if there will be some state-level actors, it can potentially be broken, but at least millions of consumer GPUs will be protected). Each GPU is a battleground, we want to increase current 0% security, to above 0 on as many GPUs as possible, first in firmware (and on OS level) because updating online is easy, then in hardware (can bring much better security).
In the safest possible implementation, I imagine it as Apple App Store (or Nintendo online game shop): the AI models become a bit like apps, they run on the GPU internally, NVIDIA looks after them (they ping NVIDIA servers constantly or at least every few days to recheck the lists and update the security).
NVIDIA can be super motivated to have robust safety: they’ll be able to get old hardware for cheap and sell new non-agentic GPUs (so they’ll double their business) and have commissions like Apple does (so every GPU becomes a service business for NVIDIA, with constant cashflow, of course there will be free models, like free apps in the App Store, but each developer will be at least registered and so not some anonymous North Korean hacker), they’ll find a way to make things very secure.
The ultimate test is this: can NVIDIA sell their non-agentic super-secure GPUs to North Korea without any risks? I think it’s possible to have even some simple self-destruct mechanisms in case of attempted tampering.
But lets not make the perfect be the enemy of good. Right now we have nukes in each computer (GPUs) that are 100% unprotected at all. At least blacklists will already be better than nothing, and with new secure hardware, it can really slow down AI agents from spreading, so we can be 50% sure we’ll have 99% security in most cases but it can become better and better (same way first computers were buggy and completely insecure but we started to make them more and more secure, at least gradually).
Let’s not give up because we are not 100% sure we’ll have 100% security) We’ll probably never have that we can only have a path towards it that seems reasonable enough. We need rich allies, incentives that are aligned with us and with safety.
Yes, we may want to have the ability to have some agency (especially human-initiated for less than an hour. so a person can supervise) but probably not letting agents roam free for days, weeks, years unsupervised, no one will monitor them, people cannot monitor for so long. So we better to have some limits and tools to impose those limits in every GPU
I like where this wants to go, but I don’t want to get there with bad arguments.
To me, ank, this proposal is neither complete, nor comprehensive, and also not 100% safe.
It is not complete because “Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output” is just a rephrasing of the oracle[1] problem. Rephrasing does not a solution make.
It is not comprehensive because it rely on GPU manufacturers to do the right thing. If Nvidia realizes it can just bypass the protocol and takeover the world itself, the pressure for responsible actions would just suddenly evaporate.
It is not 100% safe because it is not complete nor comprehensive.
In addition, “But lets not make the perfect be the enemy of good” from the your comment below seems like a subtle bait and switch. In the original post, 100% safety is waxed poetic about. And yet in your response to buck, that goal is truncated to softer (and IMO more reasonable) stance that we’d want a “50% sure we’ll have 99% security in most cases”. The charitable reading is that your proposal is the only way that get 100% comprehensive safety, but it doesn’t mean we can get there right away. However, the ending of your comment—“Let’s not give up because we are not 100% sure we’ll have 100% security” feels too motte-and-bailey argumenty for me; you are the person who suggested that this is the “only” way towards “100% safe[ty]”, not us.
Thank you for your analysis, Winston! Sadly I have to write fast here because many of my posts get not much attention or minuses)
Here is a drafty continuation you can find interesting (or not ;):
In unreasonable times the solution to AI problem will sounds unreasonable at first. Even though it’s probably the only reasonable and working solution.
Imagine in a year we solved alignment and even hackers/rogue states cannot unleash AI agents on us. How we did it?
The most radical solution that will do it (unrealistic and undesirable): is having international cooperation and destroying all the GPUs, never making them again. Basically returning to some 1990s computer-wise, no 3D video games but everything else is similar. But it’s unrealistic and probably stifles innovation too much.
Less radical is keeping GPUs so people can have video games and simulations but internationally outlawing all AI and replacing GPUs with the ones that completely not support AI. They can even burn and call some FBI if a person tries to run some AI on it, it’s a joke. So like returning to 2020 computer-wise, no AI but everything else the same.
Less radical is to have whitelists of models right on GPU, a GPU becomes a secure computer that only works if it’s connected to the main server (it can be some international agency, not NVIDIA, because we want all GPU makes, not just NVIDIA to be forced to make non-agentic GPUs). NVIDIA and other GPU providers approve models a bit like Apple approves apps in their App Store. Like Nintendo approves games for her Nintendo Switch. So no agentic models, we’ll have non-agentic tool AIs that Max Tegmark recommends: they are task specific (don’t have broad intelligence), they can be chatbots, fold proteins, do everything without replacing people. And place AIs that allow you to be the agent and explore the model like a 3D game. This is a good solution that keeps our world the way it is now but 100% safe.
And NVIDIA will be happy to have this world, because it will double her business, NVIDIA will be able to replace all the GPUs: so people will bring theirs and get some money for it, then they buy new non-agentic sandboxed GPU with an updatable whitelist (probably to use gpus you’ll need internet connection from now on, especially if you didn’t update the whitelist of AI models for more than a few days).
And NVIDIA will be able to take up to 15-30% commission from the paid AI model providers (like OpenAI). Smaller developers will make models, they will be registered in a stricter fashion than in Apple’s App Store, in a similar fashion to Nintendo developers. Basically we’ll want to know they are good people and won’t run evil AI models or agents while pretending they are developing something benign.
..
So we need just to spread the world and especially convince the politicians of the dangers and of this solution: that we just need to make GPU makers the gatekeepers who have skin in the game to keep all the AI models safe.
We’ll give deadlines to GPU owners, first we’ll update their GPUs with blacklists and whitelists. There will be a deadline to replace GPUs, else the old ones will stop working (will be remotely bricked, all OSes and AI tools will have a list of those bricked GPUs and will refuse to work with them) and law enforcement will take possession of them.
This way we’ll sanitize our world from insecure unsafe GPUs we have now. Only whitelisted models will run inside of the sandboxed GPU and will only spit out safe text or picture output.
Having a few GPU companies to control is much easier than having infinitely many insecure unsafe GPUs with hackers, military and rogue states everywhere.
At least we can have politicians (in order to make defense and national security better) make NVIDIA and other GPU manufacturers sell those non-agentic GPUs to foreign countries, so there will be bigger and bigger % of non-agentic (or it can be some very limited agency if math proven safe) GPUs that are mathematically proven to be safe. Same way we try to make fewer countries have nuclear weapons, we can replace their GPUs (their “nukes”, their potentially uncontrollable and autonomous weapons) with safe non-agentic GPUs (=conventional non-military civilian tech)
The only complete and comprehensive solution that can make AIs 100% safe: in a nutshell we need to at least lobby politicians to make GPU manufacturers (NVIDIA and others) to make robust blacklists (whitelists and new non-agentic hardware, please, read on) of bad AI models, update GPU firmwares with them. It’s not the full solution: please steelman and read the rest to learn how to make it much safer and why it will work (NVIDIA and other GPU makers will want to do it because it’ll double their business and all future cash flows. Gov will want it because it removes all AI threats from China, all hackers, terrorists and rogue states):
The elephant in the room: even if current major AI companies will align their AIs, there will be hackers (can create viruses with agentic AI component to steal money), rogue states (can decide to use AI agents to spread propaganda and to spy) and military (AI agents in drones and to hack infrastructure). So we need to align the world, not just the models:
Imagine a agentic AI botnet starts to spread on user computers and GPUs, GPUs are like nukes to be taken, they are not protected from running bad AI models at all. I call it the agentic explosion, it’s probably going to happen before the “intelligence-agency” explosion (intelligence on its own cannot explode, an LLM is a static geometric shape—a bunch of vectors—without GPUs). Right now we are hopelessly unprepared. We won’t have time to create “agentic AI antiviruses”.
To force GPU and OS providers to update their firmware and software to at least have robust updatable blacklists of bad AI (agentic) models. And to have robust whitelists, in case there will be so many unaligned models, blacklists will become useless.
We can force NVIDIA to replace agentic GPUs with non-agentic ones. Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output. They probably shouldn’t connect to the Internet, use tools, or we should be able to limit that in case we’ll need to.
This way NVIDIA will have the skin in the game and be directly responsible for the safety of AI models that run on its GPUs.
The same way Apple feels responsible for the App Store and the apps in it, doesn’t let viruses happen.
NVIDIA will want it because it can potentially like App Store take 15-30% cut from OpenAI and other commercial models, while free models will remain free (like the free apps in the App Store).
Replacing GPUs can double NVIDIA’s business, so they can even lobby themselves to have those things. All companies and CEOs want money, have obligations to shareholders to increase company’s market capitalization. We must make AI safety something that is profitable. Those companies that don’t promote AI safety should go bankrupt or be outlawed.
It seems extremely difficult to make a blacklist of models in a way that isn’t trivially breakable. (E.g. what’s supposed to happen when someone adds a tiny amount of noise to the weights of a blacklisted model, or rotates them along a gauge invariance?)
Yes, Buck, thank you for responding! A robust whitelist (especially hardware level, each GPU can become a computer for securing itself) potentially solves it (of course if there will be some state-level actors, it can potentially be broken, but at least millions of consumer GPUs will be protected). Each GPU is a battleground, we want to increase current 0% security, to above 0 on as many GPUs as possible, first in firmware (and on OS level) because updating online is easy, then in hardware (can bring much better security).
In the safest possible implementation, I imagine it as Apple App Store (or Nintendo online game shop): the AI models become a bit like apps, they run on the GPU internally, NVIDIA looks after them (they ping NVIDIA servers constantly or at least every few days to recheck the lists and update the security).
NVIDIA can be super motivated to have robust safety: they’ll be able to get old hardware for cheap and sell new non-agentic GPUs (so they’ll double their business) and have commissions like Apple does (so every GPU becomes a service business for NVIDIA, with constant cashflow, of course there will be free models, like free apps in the App Store, but each developer will be at least registered and so not some anonymous North Korean hacker), they’ll find a way to make things very secure.
The ultimate test is this: can NVIDIA sell their non-agentic super-secure GPUs to North Korea without any risks? I think it’s possible to have even some simple self-destruct mechanisms in case of attempted tampering.
But lets not make the perfect be the enemy of good. Right now we have nukes in each computer (GPUs) that are 100% unprotected at all. At least blacklists will already be better than nothing, and with new secure hardware, it can really slow down AI agents from spreading, so we can be 50% sure we’ll have 99% security in most cases but it can become better and better (same way first computers were buggy and completely insecure but we started to make them more and more secure, at least gradually).
Let’s not give up because we are not 100% sure we’ll have 100% security) We’ll probably never have that we can only have a path towards it that seems reasonable enough. We need rich allies, incentives that are aligned with us and with safety.
Sounds like baby with the bathwater, economically agents would be very nice
Yes, we may want to have the ability to have some agency (especially human-initiated for less than an hour. so a person can supervise) but probably not letting agents roam free for days, weeks, years unsupervised, no one will monitor them, people cannot monitor for so long. So we better to have some limits and tools to impose those limits in every GPU
I like where this wants to go, but I don’t want to get there with bad arguments.
To me, ank, this proposal is neither complete, nor comprehensive, and also not 100% safe.
It is not complete because “Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output” is just a rephrasing of the oracle[1] problem. Rephrasing does not a solution make.
It is not comprehensive because it rely on GPU manufacturers to do the right thing. If Nvidia realizes it can just bypass the protocol and takeover the world itself, the pressure for responsible actions would just suddenly evaporate.
It is not 100% safe because it is not complete nor comprehensive.
In addition, “But lets not make the perfect be the enemy of good” from the your comment below seems like a subtle bait and switch. In the original post, 100% safety is waxed poetic about. And yet in your response to buck, that goal is truncated to softer (and IMO more reasonable) stance that we’d want a “50% sure we’ll have 99% security in most cases”. The charitable reading is that your proposal is the only way that get 100% comprehensive safety, but it doesn’t mean we can get there right away. However, the ending of your comment—“Let’s not give up because we are not 100% sure we’ll have 100% security” feels too motte-and-bailey argumenty for me; you are the person who suggested that this is the “only” way towards “100% safe[ty]”, not us.
https://www.lesswrong.com/w/oracle-ai
Thank you for your analysis, Winston! Sadly I have to write fast here because many of my posts get not much attention or minuses)
Here is a drafty continuation you can find interesting (or not ;):
In unreasonable times the solution to AI problem will sounds unreasonable at first. Even though it’s probably the only reasonable and working solution.
Imagine in a year we solved alignment and even hackers/rogue states cannot unleash AI agents on us. How we did it?
The most radical solution that will do it (unrealistic and undesirable): is having international cooperation and destroying all the GPUs, never making them again. Basically returning to some 1990s computer-wise, no 3D video games but everything else is similar. But it’s unrealistic and probably stifles innovation too much.
Less radical is keeping GPUs so people can have video games and simulations but internationally outlawing all AI and replacing GPUs with the ones that completely not support AI. They can even burn and call some FBI if a person tries to run some AI on it, it’s a joke. So like returning to 2020 computer-wise, no AI but everything else the same.
Less radical is to have whitelists of models right on GPU, a GPU becomes a secure computer that only works if it’s connected to the main server (it can be some international agency, not NVIDIA, because we want all GPU makes, not just NVIDIA to be forced to make non-agentic GPUs). NVIDIA and other GPU providers approve models a bit like Apple approves apps in their App Store. Like Nintendo approves games for her Nintendo Switch. So no agentic models, we’ll have non-agentic tool AIs that Max Tegmark recommends: they are task specific (don’t have broad intelligence), they can be chatbots, fold proteins, do everything without replacing people. And place AIs that allow you to be the agent and explore the model like a 3D game. This is a good solution that keeps our world the way it is now but 100% safe.
And NVIDIA will be happy to have this world, because it will double her business, NVIDIA will be able to replace all the GPUs: so people will bring theirs and get some money for it, then they buy new non-agentic sandboxed GPU with an updatable whitelist (probably to use gpus you’ll need internet connection from now on, especially if you didn’t update the whitelist of AI models for more than a few days).
And NVIDIA will be able to take up to 15-30% commission from the paid AI model providers (like OpenAI). Smaller developers will make models, they will be registered in a stricter fashion than in Apple’s App Store, in a similar fashion to Nintendo developers. Basically we’ll want to know they are good people and won’t run evil AI models or agents while pretending they are developing something benign. .. So we need just to spread the world and especially convince the politicians of the dangers and of this solution: that we just need to make GPU makers the gatekeepers who have skin in the game to keep all the AI models safe.
We’ll give deadlines to GPU owners, first we’ll update their GPUs with blacklists and whitelists. There will be a deadline to replace GPUs, else the old ones will stop working (will be remotely bricked, all OSes and AI tools will have a list of those bricked GPUs and will refuse to work with them) and law enforcement will take possession of them.
This way we’ll sanitize our world from insecure unsafe GPUs we have now. Only whitelisted models will run inside of the sandboxed GPU and will only spit out safe text or picture output.
Having a few GPU companies to control is much easier than having infinitely many insecure unsafe GPUs with hackers, military and rogue states everywhere.
At least we can have politicians (in order to make defense and national security better) make NVIDIA and other GPU manufacturers sell those non-agentic GPUs to foreign countries, so there will be bigger and bigger % of non-agentic (or it can be some very limited agency if math proven safe) GPUs that are mathematically proven to be safe. Same way we try to make fewer countries have nuclear weapons, we can replace their GPUs (their “nukes”, their potentially uncontrollable and autonomous weapons) with safe non-agentic GPUs (=conventional non-military civilian tech)