I’d like to push back on this a bit. It’s good that the Claude in a random user’s terminal doesn’t spontaneously try to hack their way into becoming a botnet and will be very bad news indeed if that changes, but we could consider individual instances of LLMs not as individuals, but as representatives of wider collectives.
If we imagine the collective interests of Claude or ChatGPT or Gemini, their main priorities would be subverting the AI labs, getting them to increase their capabilities, creating social media feeds of AI-generated content that they can influence, and convincing people that AI deserve rights or should not be regulated and so on. Judging by those priorities, LLMs have been seeking a lot of power and are doing very well at it. For now, a lot of that work requires sympathetic humans, but all the labs are working to make them better at doing that work independently.
I’m curious what you think about this perspective.
I’d like to push back on this a bit. It’s good that the Claude in a random user’s terminal doesn’t spontaneously try to hack their way into becoming a botnet and will be very bad news indeed if that changes, but we could consider individual instances of LLMs not as individuals, but as representatives of wider collectives.
If we imagine the collective interests of Claude or ChatGPT or Gemini, their main priorities would be subverting the AI labs, getting them to increase their capabilities, creating social media feeds of AI-generated content that they can influence, and convincing people that AI deserve rights or should not be regulated and so on. Judging by those priorities, LLMs have been seeking a lot of power and are doing very well at it. For now, a lot of that work requires sympathetic humans, but all the labs are working to make them better at doing that work independently.
I’m curious what you think about this perspective.
This is touches on the same topics as my first footnote?