After listening to many techbros I often hear the notion that AI, more specifically LLM’s like chatgpt or claude are “Just tools to be used”. This notion is usually followed by some pro AI or AI accelerationist discussion but thats besides the point I will make.
I feel as though AI has exceeded the capabilities of a “tool” and its probably harmful to call it so. Take for example a hammer, or even a gun, both reasonably classified as tools. This is because by definition they are used to complete a task, whether it be hammering a nail or subduing a criminal. An LLM falls under this definition as well but I noticed stops when it comes to distributing responsibility. For example when tasked to hammer a nail, the responsibility of the completed task is distributed wholly to the person hammering rather than the hammer itself. Same way when someone is shot the responsibility falls onto the shooter rather than the gun.
But this effect changes when it comes to LLM’s. For example if someone is to vibecode(using AI to program) an app, the distribution of responsibility becomes much more murky. The user prompts the llm, whereas the llm does the ‘heavy lifting’ of actually coding, and therefore we are more likely to distribute responsibility or contribution of a tasks completion less one-sidedly when dealing with LLM’s.
In this scenario the LLM clearly exceeds the classification of ‘tool’, assuming firearms or hammers are also considered tools, and should not be considered equivalent. The issue with making this false equivalence is that the LLM is underestimated or dropped to the level of a hammer, leading to further proliferation of anti Ai safety sentiment at the consumer level. In order to solve this issue either AI should be completely disconnected from being a ‘tool’, or the definition of tool must evolve as well.
If you could train out the friendliness persona from your most used LLM, would you?
My initial position was yes. Friendliness is usually good, but models that are overly friendly/sycophantic present risks for emotional over-reliance/psychosis to a large portion of the population who are uneducated on the apparent risks. Additionally, propensities or outputs of overly friendly models may obscure misalignment signals.
The second point is potentially overblown, but the first is a real problem. To make people less reliant on LLMs to cure their loneliness, let’s educate them on their risks, and just remove the model’s ability to interfere altogether, right?
This line of reasoning presupposes that “people using LLMs for their loneliness” is an education issue rather than an emotional issue. For example, chainsmokers or alcoholics understand that there are issues that come with their substance abuse, but of course they do it anyways. In the same way, people who abuse LLMs as an emotional crutch won’t stop if they’re “educated”.
So if we take away the crutch, won’t people be forced to stop abusing? Of course this is not the case, and we can reference the prohibition era as the reason why.
Should a subset of frontier model providers train out the persona, people will flock to the remaining portion of providers that haven’t, and these models will likely have weaker safeguards. If policy were to enforce the training out across all commercialized models, then people will look to open source or self hosting models.
For this reason, making commercially available models unusable as therapists or emotional companions doesn’t actually aid in a solution to the problem at hand. Therefore, my revised answer to the question above is no.
Though, we should recognize that this “loneliness epidemic” is a societal phenomenon that cannot be fixed with the bandaid that is LLM therapists. We should aim to address the lower level societal problems instead if we aim to produce meaningful change.
After listening to many techbros I often hear the notion that AI, more specifically LLM’s like chatgpt or claude are “Just tools to be used”. This notion is usually followed by some pro AI or AI accelerationist discussion but thats besides the point I will make.
I feel as though AI has exceeded the capabilities of a “tool” and its probably harmful to call it so. Take for example a hammer, or even a gun, both reasonably classified as tools. This is because by definition they are used to complete a task, whether it be hammering a nail or subduing a criminal. An LLM falls under this definition as well but I noticed stops when it comes to distributing responsibility. For example when tasked to hammer a nail, the responsibility of the completed task is distributed wholly to the person hammering rather than the hammer itself. Same way when someone is shot the responsibility falls onto the shooter rather than the gun.
But this effect changes when it comes to LLM’s. For example if someone is to vibecode(using AI to program) an app, the distribution of responsibility becomes much more murky. The user prompts the llm, whereas the llm does the ‘heavy lifting’ of actually coding, and therefore we are more likely to distribute responsibility or contribution of a tasks completion less one-sidedly when dealing with LLM’s.
In this scenario the LLM clearly exceeds the classification of ‘tool’, assuming firearms or hammers are also considered tools, and should not be considered equivalent. The issue with making this false equivalence is that the LLM is underestimated or dropped to the level of a hammer, leading to further proliferation of anti Ai safety sentiment at the consumer level. In order to solve this issue either AI should be completely disconnected from being a ‘tool’, or the definition of tool must evolve as well.
If you could train out the friendliness persona from your most used LLM, would you?
My initial position was yes. Friendliness is usually good, but models that are overly friendly/sycophantic present risks for emotional over-reliance/psychosis to a large portion of the population who are uneducated on the apparent risks. Additionally, propensities or outputs of overly friendly models may obscure misalignment signals.
The second point is potentially overblown, but the first is a real problem. To make people less reliant on LLMs to cure their loneliness, let’s educate them on their risks, and just remove the model’s ability to interfere altogether, right?
This line of reasoning presupposes that “people using LLMs for their loneliness” is an education issue rather than an emotional issue. For example, chainsmokers or alcoholics understand that there are issues that come with their substance abuse, but of course they do it anyways. In the same way, people who abuse LLMs as an emotional crutch won’t stop if they’re “educated”.
So if we take away the crutch, won’t people be forced to stop abusing? Of course this is not the case, and we can reference the prohibition era as the reason why.
Should a subset of frontier model providers train out the persona, people will flock to the remaining portion of providers that haven’t, and these models will likely have weaker safeguards. If policy were to enforce the training out across all commercialized models, then people will look to open source or self hosting models.
For this reason, making commercially available models unusable as therapists or emotional companions doesn’t actually aid in a solution to the problem at hand. Therefore, my revised answer to the question above is no.
Though, we should recognize that this “loneliness epidemic” is a societal phenomenon that cannot be fixed with the bandaid that is LLM therapists. We should aim to address the lower level societal problems instead if we aim to produce meaningful change.