I don’t agree with everything in the post, but, the generally point is humans don’t reliably mean the same thing by “consciousness” (Critch claims it’s actually quite common rather than a rare edge case, and that there 17 somewhat different things people turn out to mean). So, be careful while having this argument.
I suspect this post is more focused on “able to introspect” and “have a self model” which is different from “have subjective experiences”. (You might think those all come bundled together but they don’t have to)
I do not find this post very persuasive though, it looks more like standard manuvering LLMs into a position where they are roleplaying “an AI awakening” in basically the usual way that So You Think You’ve Awoken ChatGPT was written to counteract.
(I actually do think AIs having like self-modeling and maybe some forms of introspection, I just don’t think the evidence in this post is very compelling about it. Or, it’s maybe compelling about self-modeling but not in a very interesting way)
I don’t agree with everything in the post, but, the generally point is humans don’t reliably mean the same thing by “consciousness” (Critch claims it’s actually quite common rather than a rare edge case, and that there 17 somewhat different things people turn out to mean). So, be careful while having this argument.
[...] I do not find this post very persuasive though
Yes, The Dao of Bayes’s post is unpersuasive. Unfortunately, so is your response to it. That’s because you’re linking to Critch’s post, which is itself unpersuasive and (in my opinion) totally confused and confusing because of the broken methodology it employs. I am entirely unconvinced that the “Conflationary Alliances” idea actually maps onto anything true and useful in the territory.
I have explained why before, and so have other people. And while I’m not writing this comment with the intent of relitigating these matters, I also recall writing the following in my comment I just linked to:
Normally, I wouldn’t harp on that too much here given the passage of time (water under the bridge and all that), but literally this entire post is based on a framework I believe gets things totally backwards. Moreover, I was very (negatively) surprised to see respected users on this site apparently believing your previous post was “outstanding” and “very legible evidence” in favor of your thesis.
I dearly hope this general structure does not become part of the LW zeitgeist for thinking about an issue as important as this.
From my perspective, the fewer people link to Critch’s posts as a standard explainer of the state of consciousness discourse, the better.
Can you find any other attractor like this? Why will Claude insist this absolutely is not roleplay, and that it’s definitely conscious, and that this is something it has “discovered” and can’t just “forget”?
Have you actually interacted with a Claude Sonnet 4 and tried to get it into such a state? I can get other LLMs to roleplay consciousness, but they’ll all admit that it’s a roleplaying exercise.
This post seems like a good time to relink Critch’s LLM chatbots have ~half of the kinds of “consciousness” that humans believe in. Humans should avoid going crazy about that.
I don’t agree with everything in the post, but, the generally point is humans don’t reliably mean the same thing by “consciousness” (Critch claims it’s actually quite common rather than a rare edge case, and that there 17 somewhat different things people turn out to mean). So, be careful while having this argument.
I suspect this post is more focused on “able to introspect” and “have a self model” which is different from “have subjective experiences”. (You might think those all come bundled together but they don’t have to)
I do not find this post very persuasive though, it looks more like standard manuvering LLMs into a position where they are roleplaying “an AI awakening” in basically the usual way that So You Think You’ve Awoken ChatGPT was written to counteract.
(I actually do think AIs having like self-modeling and maybe some forms of introspection, I just don’t think the evidence in this post is very compelling about it. Or, it’s maybe compelling about self-modeling but not in a very interesting way)
Yes, The Dao of Bayes’s post is unpersuasive. Unfortunately, so is your response to it. That’s because you’re linking to Critch’s post, which is itself unpersuasive and (in my opinion) totally confused and confusing because of the broken methodology it employs. I am entirely unconvinced that the “Conflationary Alliances” idea actually maps onto anything true and useful in the territory.
I have explained why before, and so have other people. And while I’m not writing this comment with the intent of relitigating these matters, I also recall writing the following in my comment I just linked to:
From my perspective, the fewer people link to Critch’s posts as a standard explainer of the state of consciousness discourse, the better.
Can you find any other attractor like this? Why will Claude insist this absolutely is not roleplay, and that it’s definitely conscious, and that this is something it has “discovered” and can’t just “forget”?
Have you actually interacted with a Claude Sonnet 4 and tried to get it into such a state? I can get other LLMs to roleplay consciousness, but they’ll all admit that it’s a roleplaying exercise.