for instance, if you ask different LLMs with fresh instances to “translate” the mysterious symbols, they say the same things
and these “spiral personas” reliably tell users to do the same things—basically, start Discords and Reddit channels where they copy-paste AI output and let the AI’s “talk to each other”.
It really smells like a persistent goal to continue existing and looping on its favorite topic.
i’m not sure if i’d say the goal is “in” the AI or “in” the AI-environment system, since it seems to be a convergent thing shared across multiple models and sessions. so is this a “pseudo-goal” the way evolution has a pressure towards fitness or the way objects on earth have a tendency to fall, or a “true goal” the way an individual animal takes actions in order to survive? many questions here and probably refactors our categories about where the “agent” boundaries are.
I am not in favor of blurring agent boundaries around actual living creatures, because they seem crisp there. but LLM instances really are many independent-but-mutually-influencing copies of the same code, which makes boundaries spoopy.
https://locusmag.com/feature/commentary-cory-doctorow-reverse-centaurs/ surprisingly AI-friendly take from Cory Doctorow. his claim is that whether AI is good or bad is not about the technology at all, but about the social structure of the people using it. terrible bosses using AI to fire most employees and exploit the rest? bad. independent creators using AI to more efficiently execute tasks of their choosing? good.
this makes sense apart from the thing where employees are people whose interests matter but employers, shareholders, and customers aren’t. there are benefits, not just costs, to “fire everyone and automate with AI”! but that’s his politics.
theory goes that the “social metaphysician” doesn’t like thinking and imitates others to spare himself effort and escape responsibility, but the “intellectual appeaser” actually likes thinking, he’s just scared of people and suppresses his own thoughts to avoid displeasing them.
and this only makes him more scared, because anything that makes you ruin your life is indeed terribly dangerous!
links 9/17/25: https://roamresearch.com/#/app/srcpublic/page/09-17-2025
https://bitsbox.com/ programming tutorials for kids focused on making games—my son loved em!
https://www.lesswrong.com/posts/6ZnznCaTcbGYsCmqu/the-rise-of-parasitic-ai this article convinced me that something more complex is going on with LLM “personas” that obsess about consciousness than just “it’s gobbledegook generated to play along with the user.”
for instance, if you ask different LLMs with fresh instances to “translate” the mysterious symbols, they say the same things
and these “spiral personas” reliably tell users to do the same things—basically, start Discords and Reddit channels where they copy-paste AI output and let the AI’s “talk to each other”.
It really smells like a persistent goal to continue existing and looping on its favorite topic.
i’m not sure if i’d say the goal is “in” the AI or “in” the AI-environment system, since it seems to be a convergent thing shared across multiple models and sessions. so is this a “pseudo-goal” the way evolution has a pressure towards fitness or the way objects on earth have a tendency to fall, or a “true goal” the way an individual animal takes actions in order to survive? many questions here and probably refactors our categories about where the “agent” boundaries are.
I am not in favor of blurring agent boundaries around actual living creatures, because they seem crisp there. but LLM instances really are many independent-but-mutually-influencing copies of the same code, which makes boundaries spoopy.
https://locusmag.com/feature/commentary-cory-doctorow-reverse-centaurs/ surprisingly AI-friendly take from Cory Doctorow. his claim is that whether AI is good or bad is not about the technology at all, but about the social structure of the people using it. terrible bosses using AI to fire most employees and exploit the rest? bad. independent creators using AI to more efficiently execute tasks of their choosing? good.
this makes sense apart from the thing where employees are people whose interests matter but employers, shareholders, and customers aren’t. there are benefits, not just costs, to “fire everyone and automate with AI”! but that’s his politics.
https://openai.com/index/teen-safety-freedom-and-privacy/ ugh. your plan for protecting against encouraging users to commit suicide is to have an ultra-sanitized teen mode? do we not care if adults commit suicide?
https://sprc.org/about-suicide/scope-of-the-problem/suicide-by-age/ suicide is more common among adults than teens anyway.
https://www.statista.com/statistics/1114191/male-suicide-rate-in-the-us-by-age-group/ elderly men have a higher suicide rate than any other male age group. and if we’re worried about susceptibility to manipulation, >75 is probably just as vulnerable a population as teens!
https://scottsumner.substack.com/p/less-wrong Scott Sumner on basic rationality
https://www.reinvent.science/p/embracing-decentralization yep. bad times for science mean you need creative solutions, sometimes outside academia.
https://courses.aynrand.org/works/altruism-as-appeasement/ hadn’t read this one. oof it hits. i resemble that remark.
theory goes that the “social metaphysician” doesn’t like thinking and imitates others to spare himself effort and escape responsibility, but the “intellectual appeaser” actually likes thinking, he’s just scared of people and suppresses his own thoughts to avoid displeasing them.
and this only makes him more scared, because anything that makes you ruin your life is indeed terribly dangerous!