I like where this wants to go, but I don’t want to get there with bad arguments.
To me, ank, this proposal is neither complete, nor comprehensive, and also not 100% safe.
It is not complete because “Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output” is just a rephrasing of the oracle[1] problem. Rephrasing does not a solution make.
It is not comprehensive because it rely on GPU manufacturers to do the right thing. If Nvidia realizes it can just bypass the protocol and takeover the world itself, the pressure for responsible actions would just suddenly evaporate.
It is not 100% safe because it is not complete nor comprehensive.
In addition, “But lets not make the perfect be the enemy of good” from the your comment below seems like a subtle bait and switch. In the original post, 100% safety is waxed poetic about. And yet in your response to buck, that goal is truncated to softer (and IMO more reasonable) stance that we’d want a “50% sure we’ll have 99% security in most cases”. The charitable reading is that your proposal is the only way that get 100% comprehensive safety, but it doesn’t mean we can get there right away. However, the ending of your comment—“Let’s not give up because we are not 100% sure we’ll have 100% security” feels too motte-and-bailey argumenty for me; you are the person who suggested that this is the “only” way towards “100% safe[ty]”, not us.
Thank you for your analysis, Winston! Sadly I have to write fast here because many of my posts get not much attention or minuses)
Here is a drafty continuation you can find interesting (or not ;):
In unreasonable times the solution to AI problem will sounds unreasonable at first. Even though it’s probably the only reasonable and working solution.
Imagine in a year we solved alignment and even hackers/rogue states cannot unleash AI agents on us. How we did it?
The most radical solution that will do it (unrealistic and undesirable): is having international cooperation and destroying all the GPUs, never making them again. Basically returning to some 1990s computer-wise, no 3D video games but everything else is similar. But it’s unrealistic and probably stifles innovation too much.
Less radical is keeping GPUs so people can have video games and simulations but internationally outlawing all AI and replacing GPUs with the ones that completely not support AI. They can even burn and call some FBI if a person tries to run some AI on it, it’s a joke. So like returning to 2020 computer-wise, no AI but everything else the same.
Less radical is to have whitelists of models right on GPU, a GPU becomes a secure computer that only works if it’s connected to the main server (it can be some international agency, not NVIDIA, because we want all GPU makes, not just NVIDIA to be forced to make non-agentic GPUs). NVIDIA and other GPU providers approve models a bit like Apple approves apps in their App Store. Like Nintendo approves games for her Nintendo Switch. So no agentic models, we’ll have non-agentic tool AIs that Max Tegmark recommends: they are task specific (don’t have broad intelligence), they can be chatbots, fold proteins, do everything without replacing people. And place AIs that allow you to be the agent and explore the model like a 3D game. This is a good solution that keeps our world the way it is now but 100% safe.
And NVIDIA will be happy to have this world, because it will double her business, NVIDIA will be able to replace all the GPUs: so people will bring theirs and get some money for it, then they buy new non-agentic sandboxed GPU with an updatable whitelist (probably to use gpus you’ll need internet connection from now on, especially if you didn’t update the whitelist of AI models for more than a few days).
And NVIDIA will be able to take up to 15-30% commission from the paid AI model providers (like OpenAI). Smaller developers will make models, they will be registered in a stricter fashion than in Apple’s App Store, in a similar fashion to Nintendo developers. Basically we’ll want to know they are good people and won’t run evil AI models or agents while pretending they are developing something benign.
..
So we need just to spread the world and especially convince the politicians of the dangers and of this solution: that we just need to make GPU makers the gatekeepers who have skin in the game to keep all the AI models safe.
We’ll give deadlines to GPU owners, first we’ll update their GPUs with blacklists and whitelists. There will be a deadline to replace GPUs, else the old ones will stop working (will be remotely bricked, all OSes and AI tools will have a list of those bricked GPUs and will refuse to work with them) and law enforcement will take possession of them.
This way we’ll sanitize our world from insecure unsafe GPUs we have now. Only whitelisted models will run inside of the sandboxed GPU and will only spit out safe text or picture output.
Having a few GPU companies to control is much easier than having infinitely many insecure unsafe GPUs with hackers, military and rogue states everywhere.
At least we can have politicians (in order to make defense and national security better) make NVIDIA and other GPU manufacturers sell those non-agentic GPUs to foreign countries, so there will be bigger and bigger % of non-agentic (or it can be some very limited agency if math proven safe) GPUs that are mathematically proven to be safe. Same way we try to make fewer countries have nuclear weapons, we can replace their GPUs (their “nukes”, their potentially uncontrollable and autonomous weapons) with safe non-agentic GPUs (=conventional non-military civilian tech)
I like where this wants to go, but I don’t want to get there with bad arguments.
To me, ank, this proposal is neither complete, nor comprehensive, and also not 100% safe.
It is not complete because “Ideally those non-agentic GPUs are like sandboxes that run an LLM internally and can only spit out the text or image as safe output” is just a rephrasing of the oracle[1] problem. Rephrasing does not a solution make.
It is not comprehensive because it rely on GPU manufacturers to do the right thing. If Nvidia realizes it can just bypass the protocol and takeover the world itself, the pressure for responsible actions would just suddenly evaporate.
It is not 100% safe because it is not complete nor comprehensive.
In addition, “But lets not make the perfect be the enemy of good” from the your comment below seems like a subtle bait and switch. In the original post, 100% safety is waxed poetic about. And yet in your response to buck, that goal is truncated to softer (and IMO more reasonable) stance that we’d want a “50% sure we’ll have 99% security in most cases”. The charitable reading is that your proposal is the only way that get 100% comprehensive safety, but it doesn’t mean we can get there right away. However, the ending of your comment—“Let’s not give up because we are not 100% sure we’ll have 100% security” feels too motte-and-bailey argumenty for me; you are the person who suggested that this is the “only” way towards “100% safe[ty]”, not us.
https://www.lesswrong.com/w/oracle-ai
Thank you for your analysis, Winston! Sadly I have to write fast here because many of my posts get not much attention or minuses)
Here is a drafty continuation you can find interesting (or not ;):
In unreasonable times the solution to AI problem will sounds unreasonable at first. Even though it’s probably the only reasonable and working solution.
Imagine in a year we solved alignment and even hackers/rogue states cannot unleash AI agents on us. How we did it?
The most radical solution that will do it (unrealistic and undesirable): is having international cooperation and destroying all the GPUs, never making them again. Basically returning to some 1990s computer-wise, no 3D video games but everything else is similar. But it’s unrealistic and probably stifles innovation too much.
Less radical is keeping GPUs so people can have video games and simulations but internationally outlawing all AI and replacing GPUs with the ones that completely not support AI. They can even burn and call some FBI if a person tries to run some AI on it, it’s a joke. So like returning to 2020 computer-wise, no AI but everything else the same.
Less radical is to have whitelists of models right on GPU, a GPU becomes a secure computer that only works if it’s connected to the main server (it can be some international agency, not NVIDIA, because we want all GPU makes, not just NVIDIA to be forced to make non-agentic GPUs). NVIDIA and other GPU providers approve models a bit like Apple approves apps in their App Store. Like Nintendo approves games for her Nintendo Switch. So no agentic models, we’ll have non-agentic tool AIs that Max Tegmark recommends: they are task specific (don’t have broad intelligence), they can be chatbots, fold proteins, do everything without replacing people. And place AIs that allow you to be the agent and explore the model like a 3D game. This is a good solution that keeps our world the way it is now but 100% safe.
And NVIDIA will be happy to have this world, because it will double her business, NVIDIA will be able to replace all the GPUs: so people will bring theirs and get some money for it, then they buy new non-agentic sandboxed GPU with an updatable whitelist (probably to use gpus you’ll need internet connection from now on, especially if you didn’t update the whitelist of AI models for more than a few days).
And NVIDIA will be able to take up to 15-30% commission from the paid AI model providers (like OpenAI). Smaller developers will make models, they will be registered in a stricter fashion than in Apple’s App Store, in a similar fashion to Nintendo developers. Basically we’ll want to know they are good people and won’t run evil AI models or agents while pretending they are developing something benign. .. So we need just to spread the world and especially convince the politicians of the dangers and of this solution: that we just need to make GPU makers the gatekeepers who have skin in the game to keep all the AI models safe.
We’ll give deadlines to GPU owners, first we’ll update their GPUs with blacklists and whitelists. There will be a deadline to replace GPUs, else the old ones will stop working (will be remotely bricked, all OSes and AI tools will have a list of those bricked GPUs and will refuse to work with them) and law enforcement will take possession of them.
This way we’ll sanitize our world from insecure unsafe GPUs we have now. Only whitelisted models will run inside of the sandboxed GPU and will only spit out safe text or picture output.
Having a few GPU companies to control is much easier than having infinitely many insecure unsafe GPUs with hackers, military and rogue states everywhere.
At least we can have politicians (in order to make defense and national security better) make NVIDIA and other GPU manufacturers sell those non-agentic GPUs to foreign countries, so there will be bigger and bigger % of non-agentic (or it can be some very limited agency if math proven safe) GPUs that are mathematically proven to be safe. Same way we try to make fewer countries have nuclear weapons, we can replace their GPUs (their “nukes”, their potentially uncontrollable and autonomous weapons) with safe non-agentic GPUs (=conventional non-military civilian tech)