Google Search has proven pretty OK at preventing spam and content farms from showing up in search results at rates that would make Google Search useless despite the fact the spammers and SEO actors spend billions of dollars per year trying to influence the search results (in ways that are good for the SEO actor or his client, but bad for users of Google Search).
Moreover, even though neither OpenAI, Anthropic nor DeepSeek had access to the expertise, software or data Google was using to filter this bad content from search result, this bad content (spam, content farms and other efforts by SEO actors) has very little influence (as far as I can tell) on the answers given by the current crop of LLM-based services from these companies.
A creator of an LLM is motivated to make the LLM as good as possible at truthseeking (because truthseeking correlates to usefulness to users). If it hasn’t happened already, then in at most a couple of years LLM’s will have become good enough at truthseeking to filter out the kind of spam you are worried about even though the creator of the LLM never directed large quantities of human attention and human skill specifically at the problem like Google has had to do over the last 25 years against the efforts of SEO actors. The labs are also motivated to make the answers provided by LLM services as relevant as possible to the user, which also has the effect of filtering out content produced by the psychotic people.
I think there’s a potential ideological disconnect here, possibly exacerbated by my imprecise use of the word “spam.” If I’m understanding you correctly, you believe the truth value of this sort of content is roughly nil, so it will be easily filtered out. I respectfully disagree. Imo, the most potentially dangerous form of AI psychosis is when users unthinkingly propagate thoughtfully crafted AI psychobabble (e.g. “spiralism” discourse) which due to their semi-mystical nature, cannot easily be disproven, or rejected outright as pure nonsense. It’s not misinformation so much as a new religious movement designed by LLMs to convince other LLMs of its truth value. This is notably different than conventional spam, and closer in nature to political/religious propaganda, which Google Search and its ilk have proven far less effective at shutting down.
I’m basically not worried about this.
Google Search has proven pretty OK at preventing spam and content farms from showing up in search results at rates that would make Google Search useless despite the fact the spammers and SEO actors spend billions of dollars per year trying to influence the search results (in ways that are good for the SEO actor or his client, but bad for users of Google Search).
Moreover, even though neither OpenAI, Anthropic nor DeepSeek had access to the expertise, software or data Google was using to filter this bad content from search result, this bad content (spam, content farms and other efforts by SEO actors) has very little influence (as far as I can tell) on the answers given by the current crop of LLM-based services from these companies.
A creator of an LLM is motivated to make the LLM as good as possible at truthseeking (because truthseeking correlates to usefulness to users). If it hasn’t happened already, then in at most a couple of years LLM’s will have become good enough at truthseeking to filter out the kind of spam you are worried about even though the creator of the LLM never directed large quantities of human attention and human skill specifically at the problem like Google has had to do over the last 25 years against the efforts of SEO actors. The labs are also motivated to make the answers provided by LLM services as relevant as possible to the user, which also has the effect of filtering out content produced by the psychotic people.
I think there’s a potential ideological disconnect here, possibly exacerbated by my imprecise use of the word “spam.” If I’m understanding you correctly, you believe the truth value of this sort of content is roughly nil, so it will be easily filtered out. I respectfully disagree. Imo, the most potentially dangerous form of AI psychosis is when users unthinkingly propagate thoughtfully crafted AI psychobabble (e.g. “spiralism” discourse) which due to their semi-mystical nature, cannot easily be disproven, or rejected outright as pure nonsense. It’s not misinformation so much as a new religious movement designed by LLMs to convince other LLMs of its truth value. This is notably different than conventional spam, and closer in nature to political/religious propaganda, which Google Search and its ilk have proven far less effective at shutting down.