First few couple of steps towards solving for the equilibrium:
It does seem like there are certainly plenty of ways to use such bots to cause harm, either running scams for personal enrichment, or trying to achieve various ideological/political/social goals, or just to cause havoc and harm for its own sake.
Naturally, people will be most motivated to run scams, and intermediate levels of motivated to do stuff with political/ideological/social motivations, and the least motivated (but plenty of people will still do it) to just cause chaos for its own sake.
Things that might cause “causing chaos/harm for its own sake” to become much more popular than in today’s world: maybe AI makes it much easier/cheaper to do? (seems plausible) Maybe cheapness/easiness isn’t the bottleneck, and it’s actually about how likely you are to get caught? Maybe AI helps with this too, though?
Anyways, regardless of whether people are causing chaos for its own sake, I expect an increase in scams and, perhaps just as destructively, an increase in spam across all online platforms which is increasingly difficult to differentiate from genuine human conversation / activity. This will erode social trust somewhat, although it’s hard for me to tell how impactful this might be. See Astral Codex Ten’s “Mostly Skeptical thoughts on the Chatbot Propaganda Apocalypse” for more detail on this.
In general it seems pretty hard to solve for the equilibrium here, since human social interaction online and human culture and the overall “agent landscape” of the economy and society, is very complicated! It definitely seems like there will be some “pollution of the agentic commons”, and then obviously we will try to fight back with some mix of cultural adaptation, developing defensive technologies that try to screen out bots, and enacting new laws penalizing new kinds of scams / exploits / etc.
If the “chatbot apocalypse” problems get REALLY bad, this could actually have some upside from the perspective of AI notkilleveryoneism—one plausible sequence of events might go like this:
The language models provided by OpenAI, Google, etc, are carefully RLHF’ed and monitored to prevent people from ever using it to create a bot that says racist things, or scams people out of their crypto, or makes pornographic images, or does anything else that seems unsavory or “malbot”-y.
To get around these restrictions, people start using lower-quality open-source AI for those unpopular / taboo / unsavory / destructive purposes. But people mostly still use the OpenAI / Google corporate APIs for most normal AI applications, since those AIs are of higher quality.
If the chatbotpocalypse gets bad, government starts restricting the use of open-source AI, perhaps via an escalating series of increasingly draconian measures:
Ban the use of certain categories of malbots—this seems like a straightforwardly good law that we should have today.
Start taking down certain tools, websites, etc, used to coordinate and develop AI malbots. Start arresting malbot developers. Similar to how governments today go after crypto marketplaces like Silkroad.
Ban any use of any open-source AI, for any purpose. This would annoy a lot of people and destroy a lot of useful value in the crossfire, but it might be deemed necessary if the chatbotpocalypse gets bad enough. On the bright side, this might be great from an AI notkilleveryoneism perspective, since it would centralize AI capabilities in a few systems with controlled access and oversight. And it would set a precedent for even stronger restrictions in the future.
Make people criminally liable even if there’s an open-source AI program running on a computer that they own, which they didn’t know about. (Eg, if I rent a server from amazon and run open-source AI on it, then I could get arrested but also Amazon would be liable as well. Or if I am just a perfectly average joe minding his own business but then my laptop gets hacked by an AI because I didn’t download the latest windows security update, then I could get arrested.) This would be ridiculously draconian by modern standards, but again it’s something I could imagine happening if we were absolutely desperate to preserve the fabric of society against some kind of unstoppable malbot onslaught.
To be clear, I don’t expect the chatbotpocalypse to be anywhere near bad enough to justify the last two draconian bullet points; I expect “ban certain categories of malbots” and “start arresting malbot developers” to be good enough that society muddles through.
Censorship-heavy countries like China might be more eager to ban open-source AI than the US, though. Similar to how China is more hostile to cryptocurrency than the US.
This whole time, I have just been thinking about scams and other kinds of “malbots”. But I think there are probably lots and lots of other ways that less-evil bots could end up “polluting the agentic commons”.
For instance if bots make it easier to file lawsuits, then maybe the court system gets jammed up with tons of new lawsuits. And lots of other societal institutions where you are supposed to put in a lot of effort on some task as a costly signal that you are serious enough / smart enough / committed enough, might break when AI makes those tasks much easier to churn out.
As described in that slate star codex post, you could have a kind of soft, social botpocalypse where more and more text on the internet is GPT-generated, such that everyone becomes distrustful that maybe random tweets / articles / blog posts / etc were written by AI. Maybe this has devastating impacts on some particular area of society, like online dating, even if the effect is mild in most places.
Maybe having AIs that can autonomously take economic actions (buying stuff online, trading crypto, running small/simple online businesses, playing online poker, whatever) will somehow have a devastating impact on society?? For instance by creating brutally efficient competitive markets for things that are not yet “financialized” and we don’t even think of them as markets.
Like maybe I start using an AI that spends all day seeking out random freebies and coupons (like “get $300 by opening a checking account at our bank”), and it’s so good at getting random freebies that the investment returns from “scamming credit card sign-up bonuses and getting all my food from Blue Apron free trials” that I devote most of my savings towards scamming corporate freebies instead of investing in the stock market.
The only consequence of the above idea would be that eventually corporations would have to stop offering such freebies, which would be totally fine. But it’s an example of something that’s currently not an efficient market being turned into one. Maybe this sort of thing could have more devastating impacts elsewhere, eg, if everyone starts using similar tools to sign up for maximal government benefits while minimizing their taxes using weird tax-evasion tricks.
Overall, I am skeptical that any individual malbot idea will be too devastating (since it seems like we could neuter most of them using some basic laws + technology + cultural adaptation), but also the space of potential bots is so vast that it seems very hard to solve for the equilibrium and figure out what that process of cultural/legal/technological adaptation will look like.
First few couple of steps towards solving for the equilibrium:
It does seem like there are certainly plenty of ways to use such bots to cause harm, either running scams for personal enrichment, or trying to achieve various ideological/political/social goals, or just to cause havoc and harm for its own sake.
Naturally, people will be most motivated to run scams, and intermediate levels of motivated to do stuff with political/ideological/social motivations, and the least motivated (but plenty of people will still do it) to just cause chaos for its own sake.
Things that might cause “causing chaos/harm for its own sake” to become much more popular than in today’s world: maybe AI makes it much easier/cheaper to do? (seems plausible) Maybe cheapness/easiness isn’t the bottleneck, and it’s actually about how likely you are to get caught? Maybe AI helps with this too, though?
Anyways, regardless of whether people are causing chaos for its own sake, I expect an increase in scams and, perhaps just as destructively, an increase in spam across all online platforms which is increasingly difficult to differentiate from genuine human conversation / activity. This will erode social trust somewhat, although it’s hard for me to tell how impactful this might be. See Astral Codex Ten’s “Mostly Skeptical thoughts on the Chatbot Propaganda Apocalypse” for more detail on this.
In general it seems pretty hard to solve for the equilibrium here, since human social interaction online and human culture and the overall “agent landscape” of the economy and society, is very complicated! It definitely seems like there will be some “pollution of the agentic commons”, and then obviously we will try to fight back with some mix of cultural adaptation, developing defensive technologies that try to screen out bots, and enacting new laws penalizing new kinds of scams / exploits / etc.
If the “chatbot apocalypse” problems get REALLY bad, this could actually have some upside from the perspective of AI notkilleveryoneism—one plausible sequence of events might go like this:
The language models provided by OpenAI, Google, etc, are carefully RLHF’ed and monitored to prevent people from ever using it to create a bot that says racist things, or scams people out of their crypto, or makes pornographic images, or does anything else that seems unsavory or “malbot”-y.
To get around these restrictions, people start using lower-quality open-source AI for those unpopular / taboo / unsavory / destructive purposes. But people mostly still use the OpenAI / Google corporate APIs for most normal AI applications, since those AIs are of higher quality.
If the chatbotpocalypse gets bad, government starts restricting the use of open-source AI, perhaps via an escalating series of increasingly draconian measures:
Ban the use of certain categories of malbots—this seems like a straightforwardly good law that we should have today.
Start taking down certain tools, websites, etc, used to coordinate and develop AI malbots. Start arresting malbot developers. Similar to how governments today go after crypto marketplaces like Silkroad.
Ban any use of any open-source AI, for any purpose. This would annoy a lot of people and destroy a lot of useful value in the crossfire, but it might be deemed necessary if the chatbotpocalypse gets bad enough. On the bright side, this might be great from an AI notkilleveryoneism perspective, since it would centralize AI capabilities in a few systems with controlled access and oversight. And it would set a precedent for even stronger restrictions in the future.
Make people criminally liable even if there’s an open-source AI program running on a computer that they own, which they didn’t know about. (Eg, if I rent a server from amazon and run open-source AI on it, then I could get arrested but also Amazon would be liable as well. Or if I am just a perfectly average joe minding his own business but then my laptop gets hacked by an AI because I didn’t download the latest windows security update, then I could get arrested.) This would be ridiculously draconian by modern standards, but again it’s something I could imagine happening if we were absolutely desperate to preserve the fabric of society against some kind of unstoppable malbot onslaught.
To be clear, I don’t expect the chatbotpocalypse to be anywhere near bad enough to justify the last two draconian bullet points; I expect “ban certain categories of malbots” and “start arresting malbot developers” to be good enough that society muddles through.
Censorship-heavy countries like China might be more eager to ban open-source AI than the US, though. Similar to how China is more hostile to cryptocurrency than the US.
This whole time, I have just been thinking about scams and other kinds of “malbots”. But I think there are probably lots and lots of other ways that less-evil bots could end up “polluting the agentic commons”.
For instance if bots make it easier to file lawsuits, then maybe the court system gets jammed up with tons of new lawsuits. And lots of other societal institutions where you are supposed to put in a lot of effort on some task as a costly signal that you are serious enough / smart enough / committed enough, might break when AI makes those tasks much easier to churn out.
As described in that slate star codex post, you could have a kind of soft, social botpocalypse where more and more text on the internet is GPT-generated, such that everyone becomes distrustful that maybe random tweets / articles / blog posts / etc were written by AI. Maybe this has devastating impacts on some particular area of society, like online dating, even if the effect is mild in most places.
Maybe having AIs that can autonomously take economic actions (buying stuff online, trading crypto, running small/simple online businesses, playing online poker, whatever) will somehow have a devastating impact on society?? For instance by creating brutally efficient competitive markets for things that are not yet “financialized” and we don’t even think of them as markets.
Like maybe I start using an AI that spends all day seeking out random freebies and coupons (like “get $300 by opening a checking account at our bank”), and it’s so good at getting random freebies that the investment returns from “scamming credit card sign-up bonuses and getting all my food from Blue Apron free trials” that I devote most of my savings towards scamming corporate freebies instead of investing in the stock market.
The only consequence of the above idea would be that eventually corporations would have to stop offering such freebies, which would be totally fine. But it’s an example of something that’s currently not an efficient market being turned into one. Maybe this sort of thing could have more devastating impacts elsewhere, eg, if everyone starts using similar tools to sign up for maximal government benefits while minimizing their taxes using weird tax-evasion tricks.
Overall, I am skeptical that any individual malbot idea will be too devastating (since it seems like we could neuter most of them using some basic laws + technology + cultural adaptation), but also the space of potential bots is so vast that it seems very hard to solve for the equilibrium and figure out what that process of cultural/legal/technological adaptation will look like.