The problems with AI: a thoughtful analysis from someone who is obsessed with AI.
I can’t stop thinking about AI. How it’s made, what it is, what a person is, what I could use it for right now, what I dream of using it for in a year, how it has affected my society, whether we will create a superintelligence, and really the list keeps going.
People frequently say to me: “AI is immoral, and therefore I don’t use it” and some go so far as “Stop using it right now”. Since I value social validation, and the volume of complaints is high enough, I cannot shrug off this negative feedback. I need to either stop using AI or to write down my stance.
Why not quit? Well. In addition to being obsessed I think I would be fired. My wife is also similarly obsessed with AI, and quitting would put a lot of tension between us. If I lost my obsession, lost my job, and damaged my marriage, that would cause me sufficient harm to outweigh any moral benefits from quitting.
Thus necessitates a stance. A self defense mechanism? It is clear a weak stance will not do. It will need to support the weight of my entire life. This writeup is scoping out the terrain for which it may be possible for me to construct such a stance. Critique and reader feedback should be essential before progressing further.
So. Why might AI be immoral? I divide the reasons into two groups.
Group 1: AI is made by an unethical process
Group 2: AI output is unethical
Objection 1. AI is made by an unethical process.
Suppose I were to learn that a popular company uses slave labor to make their product. I would then immediately boycott the products that company produces, as well as actively speak against it to my peers.
Objection 1A. AI is pollution.
We observe extremely harsh environmental impact from existing data centers. They are forced upon low income communities, often racial minorities. They consume vast amounts of electricity, raising the price of electricity for locals. They produce vast amounts of heat, often using vast amounts of water for cooling. They contribute to global warming. They take up land, produce sound, and are generally unpleasant.
It seems to me that the scale of these effects needs to be quantified. Unfortunately, I find doing research on the subject extremely difficult because of propaganda. I don’t trust AI or reddit or really much of anything to give me an unbiased indication of the scale of the problem.
I think my request is essentially to compare the environmental destruction caused by AI to that caused by cars. How does buying a car compare to buying a subscription to AI? How does driving over taking the bus compare to doing a coding project with AI vs by hand? How do car companies and AI companies impact the environment overall?
I don’t want to shy away from tail risk here. I recall that the car industry resulted in a global lead poisoning pandemic. Who knows what damage we might cause in the pursuit of intelligent AI.
And finally, I’d like to evaluate the benefits to society gained from the car industry. Is it plausible that AI can have similar benefits? Not necessarily to dismiss the costs, but it would be a bit ridiculous to tell your peers to stop using cars in the USA today.
Objection 1B. AI is stealing.
AI companies have illegally pirated billions of dollars of books to train the language models. AI text and images often plagiarize content, offering no attribution to where the content came from originally. Authors have no say in this process, and no way to gain compensation for the AI output.
AI companies should at least have to buy a book in order to train a model with it. I don’t know how to draw the line between a person buying a book and writing a book inspired by it, vs an AI adapting the content of the book. In some sense human output is determined by a bunch of input generated from other humans. As an author, isn’t the point of writing a book so that people will read it and feel inspired?
Objection 2. AI output is unethical.
It would be better if the prompts were left unanswered. I’m going to divide these objections into mutually exclusive categories based on how powerful the AI turns out to be.
2A. AI is slop.
2B. AI is poison.
2C. AI is power.
2D. AI is unemployment.
2E. AI is extinction.
Fully asserting that AI is slop and that AI is poison should be contradictory, however note that the position (Either 2A or 2B or 2C) is fully reasonable. I haven’t heard anyone say something like that though, mostly people just pick whichever seems the most likely to them.
Objection 2A. AI is slop.
The capabilities of AI are constantly overexaggerated, overhyped, and unfounded. AI companies use this to profit from low quality products at the expense of their consumers. AI text reads like empty words, AI images distract us through manipulation of extremely baseline human instincts. The internet is becoming useless due to the volume of low quality AI content. Eventually the problem will compound when AI starts training on its own outputs. AI consumes a large amount of resources and provides nothing of value to society in return, leaving a distasteful mark on everything it touches.
I find this argument… uncompelling. Prompting AI for text feels me with a sense of awe, using it to code with multiple agents feels me with a sense of power. Even if others are inhaling shit, I’m personally doing fine. Also my coworkers and spouse share my sentiment.
Objection 2B. AI is poison.
AI is poisoning our education system. Making it easier to cheat will cause more cheating. This will disproportionately harm weaker students that are more likely to cheat. It is also collectively working to make humanity dumber. By offering to think for people, people think less. People are already trusting AI more than other humans, indicating a breakdown of critical analysis.
To this I say: Yep farm equipment made farmers have weaker muscles. I am 100% confident that I have gotten worse at coding since Claude Code came out. I’m also proud of it. Coding isn’t inherently interesting to me. Having programs solve problems for me is what’s interesting.
Secondly, I believe that education needs to adapt to the times. We should be teaching people how to navigate a world armed with AI, regardless of whether it exists yet. Banning AI will reinforce the idea that AI is cheating, rather than teaching people that AI is a tool that can be used to avoid learning the details of something. There should be a class on building nice webapps that entirely uses AI and there should be a different class on HTML where people learn how to write HTML with very strict no cheating policies. Students will sign up for whatever they want to learn. In no sense is learning HTML an objectively important skill for everyone.
Objection 2C. AI is power.
Deepfakes, scamming, homemade bombs, blackmail. Bad actors are getting a major boost in easily grabbed weapons. AI empowers those with access to it, and it is easy to foresee this going very wrong. Those with more access to AI are able to use it to consolidate power and deny access to others.
These are real problems that we need to think about. It’s a little unclear that slowing down the top AI companies will make us safer from this generally speaking, but definitely in the short term there could be benefits.
Objection 2D. AI is unemployment.
AI is going to have a huge impact on hiring. This is going to economically devastate entire industries, putting working class families at risk.
I think it’s nice that people won’t need to do mindless jobs. What we need is not to stop the automation of jobs, what we need to do is support people as their lives change due to technology. In the case of the art industry, I am not opposed to completely de-monetizing art. It can be a hobby like gardening, and there absolutely will be some demand for lovely homemade art too.
Objection 2E. AI is extinction.
People are already starting to hook up AI to the controls of weapons. It’s not that hard to visualize a superintelligent AI that has influence over the real world. If it decides to cheat its training data by killing all humans, then the stakes are immediately raised to the max. Should it succeed in outsmarting us, the game is over forever.
I have read some articles on Less Wrong about this possibility recently. I don’t see the nonzero risk of extinction as a reason not to build a superintelligence. When we imagine science fiction scenarios, who can assign ethical values to the possible outcomes?
The problems with AI: a thoughtful analysis from someone who is obsessed with AI.
I can’t stop thinking about AI. How it’s made, what it is, what a person is, what I could use it for right now, what I dream of using it for in a year, how it has affected my society, whether we will create a superintelligence, and really the list keeps going.
People frequently say to me: “AI is immoral, and therefore I don’t use it” and some go so far as “Stop using it right now”. Since I value social validation, and the volume of complaints is high enough, I cannot shrug off this negative feedback. I need to either stop using AI or to write down my stance.
Why not quit? Well. In addition to being obsessed I think I would be fired. My wife is also similarly obsessed with AI, and quitting would put a lot of tension between us. If I lost my obsession, lost my job, and damaged my marriage, that would cause me sufficient harm to outweigh any moral benefits from quitting.
Thus necessitates a stance. A self defense mechanism? It is clear a weak stance will not do. It will need to support the weight of my entire life. This writeup is scoping out the terrain for which it may be possible for me to construct such a stance. Critique and reader feedback should be essential before progressing further.
So. Why might AI be immoral? I divide the reasons into two groups.
Group 1: AI is made by an unethical process
Group 2: AI output is unethical
Objection 1. AI is made by an unethical process.
Suppose I were to learn that a popular company uses slave labor to make their product. I would then immediately boycott the products that company produces, as well as actively speak against it to my peers.
Objection 1A. AI is pollution.
We observe extremely harsh environmental impact from existing data centers. They are forced upon low income communities, often racial minorities. They consume vast amounts of electricity, raising the price of electricity for locals. They produce vast amounts of heat, often using vast amounts of water for cooling. They contribute to global warming. They take up land, produce sound, and are generally unpleasant.
It seems to me that the scale of these effects needs to be quantified. Unfortunately, I find doing research on the subject extremely difficult because of propaganda. I don’t trust AI or reddit or really much of anything to give me an unbiased indication of the scale of the problem.
I think my request is essentially to compare the environmental destruction caused by AI to that caused by cars. How does buying a car compare to buying a subscription to AI? How does driving over taking the bus compare to doing a coding project with AI vs by hand? How do car companies and AI companies impact the environment overall?
I don’t want to shy away from tail risk here. I recall that the car industry resulted in a global lead poisoning pandemic. Who knows what damage we might cause in the pursuit of intelligent AI.
And finally, I’d like to evaluate the benefits to society gained from the car industry. Is it plausible that AI can have similar benefits? Not necessarily to dismiss the costs, but it would be a bit ridiculous to tell your peers to stop using cars in the USA today.
Objection 1B. AI is stealing.
AI companies have illegally pirated billions of dollars of books to train the language models. AI text and images often plagiarize content, offering no attribution to where the content came from originally. Authors have no say in this process, and no way to gain compensation for the AI output.
AI companies should at least have to buy a book in order to train a model with it. I don’t know how to draw the line between a person buying a book and writing a book inspired by it, vs an AI adapting the content of the book. In some sense human output is determined by a bunch of input generated from other humans. As an author, isn’t the point of writing a book so that people will read it and feel inspired?
Objection 2. AI output is unethical.
It would be better if the prompts were left unanswered. I’m going to divide these objections into mutually exclusive categories based on how powerful the AI turns out to be.
2A. AI is slop.
2B. AI is poison.
2C. AI is power.
2D. AI is unemployment.
2E. AI is extinction.
Fully asserting that AI is slop and that AI is poison should be contradictory, however note that the position (Either 2A or 2B or 2C) is fully reasonable. I haven’t heard anyone say something like that though, mostly people just pick whichever seems the most likely to them.
Objection 2A. AI is slop.
The capabilities of AI are constantly overexaggerated, overhyped, and unfounded. AI companies use this to profit from low quality products at the expense of their consumers. AI text reads like empty words, AI images distract us through manipulation of extremely baseline human instincts. The internet is becoming useless due to the volume of low quality AI content. Eventually the problem will compound when AI starts training on its own outputs. AI consumes a large amount of resources and provides nothing of value to society in return, leaving a distasteful mark on everything it touches.
I find this argument… uncompelling. Prompting AI for text feels me with a sense of awe, using it to code with multiple agents feels me with a sense of power. Even if others are inhaling shit, I’m personally doing fine. Also my coworkers and spouse share my sentiment.
Objection 2B. AI is poison.
AI is poisoning our education system. Making it easier to cheat will cause more cheating. This will disproportionately harm weaker students that are more likely to cheat. It is also collectively working to make humanity dumber. By offering to think for people, people think less. People are already trusting AI more than other humans, indicating a breakdown of critical analysis.
To this I say: Yep farm equipment made farmers have weaker muscles. I am 100% confident that I have gotten worse at coding since Claude Code came out. I’m also proud of it. Coding isn’t inherently interesting to me. Having programs solve problems for me is what’s interesting.
Secondly, I believe that education needs to adapt to the times. We should be teaching people how to navigate a world armed with AI, regardless of whether it exists yet. Banning AI will reinforce the idea that AI is cheating, rather than teaching people that AI is a tool that can be used to avoid learning the details of something. There should be a class on building nice webapps that entirely uses AI and there should be a different class on HTML where people learn how to write HTML with very strict no cheating policies. Students will sign up for whatever they want to learn. In no sense is learning HTML an objectively important skill for everyone.
Objection 2C. AI is power.
Deepfakes, scamming, homemade bombs, blackmail. Bad actors are getting a major boost in easily grabbed weapons. AI empowers those with access to it, and it is easy to foresee this going very wrong. Those with more access to AI are able to use it to consolidate power and deny access to others.
These are real problems that we need to think about. It’s a little unclear that slowing down the top AI companies will make us safer from this generally speaking, but definitely in the short term there could be benefits.
Objection 2D. AI is unemployment.
AI is going to have a huge impact on hiring. This is going to economically devastate entire industries, putting working class families at risk.
I think it’s nice that people won’t need to do mindless jobs. What we need is not to stop the automation of jobs, what we need to do is support people as their lives change due to technology. In the case of the art industry, I am not opposed to completely de-monetizing art. It can be a hobby like gardening, and there absolutely will be some demand for lovely homemade art too.
Objection 2E. AI is extinction.
People are already starting to hook up AI to the controls of weapons. It’s not that hard to visualize a superintelligent AI that has influence over the real world. If it decides to cheat its training data by killing all humans, then the stakes are immediately raised to the max. Should it succeed in outsmarting us, the game is over forever.
I have read some articles on Less Wrong about this possibility recently. I don’t see the nonzero risk of extinction as a reason not to build a superintelligence. When we imagine science fiction scenarios, who can assign ethical values to the possible outcomes?