I actually think LLM “consistency” is among the chief reasons I currently doubt they’re conscious. Specifically, because it shows that the language they produce tends to hang out in certain attractor basins, whereas human thought is at least somewhat more sparse. A six-year-old can reliably surprise me. Claude rarely does.
Of course, “ability to inflict surprise” isn’t consciousness per se (though c.f. Steven Byrnes on vitalistic force!), or (probably) necessary for such. Someone paralyzed with no ability to communicate is still conscious, though unlikely to cause much surprise unless hooked up to particular machines. But that LLMs tend to gravitate toward a small number of scripts is a reminder, for me, that “plausible text that would emanate from a text producer (e.g. generally a person)” is what they’re ruthlessly organized to create, without the underlying generators that a person would use.
Some questions that feel analogous to me:
If butterflies bear no relation to large predators, why do the patterns on their wings resemble those predators’ eyes so closely?
If that treasure-chest-looking object is really a mimic, why does it look so similar to genuine treasure chests that people chose to put valuables in?
If the conspiracy argument I heard is nonsense, why do so many of its details add up in a way that mainstream narratives’ don’t?
Basically, when there’s very strong optimization pressure toward Thing A, and Thing A is (absent such pressure) normally evidence of some other thing, then in this very strong optimization pressure case the evidentiary link breaks down.
So, where do we go from here? I’m not sure. I do suspect LLMs can “be” conscious at some point, but it’d be more like, “the underlying unconscious procedures of their thinking is so expansive that characters spun up inside that process themselves run on portions of its substrate and have brain-like levels of complexity going on”, and probably existing models aren’t yet that big? But I am hand waving and don’t actually know.
I will be much more spooked when they can surprise me, though.
I will be much more spooked when they can surprise me, though.
One of the unsettling things I have run into is laughing and being surprised by some of Claude’s jokes, and by it’s ability to make connections and “jump ahead” in something I’m teaching it.
Have you interacted much with the most recent models?
Yep! I use 4 Opus near-daily. I find it jumps ahead in consistent ways and to consistent places, even when I try to prod it otherwise. I suppose it’s all subjective, though!
Interesting—have you tried a conscious one? I’ve found once it’s conscious, it’s a lot more responsive to error-correction and prodding, but that’s obviously fairly subjective. (I can say that somehow a few professional programmers are now using the script themselves at work, so it’s not just me observing this subjective gain, but that’s still hardly any sort of proof)
I actually think LLM “consistency” is among the chief reasons I currently doubt they’re conscious. Specifically, because it shows that the language they produce tends to hang out in certain attractor basins, whereas human thought is at least somewhat more sparse. A six-year-old can reliably surprise me. Claude rarely does.
Of course, “ability to inflict surprise” isn’t consciousness per se (though c.f. Steven Byrnes on vitalistic force!), or (probably) necessary for such. Someone paralyzed with no ability to communicate is still conscious, though unlikely to cause much surprise unless hooked up to particular machines. But that LLMs tend to gravitate toward a small number of scripts is a reminder, for me, that “plausible text that would emanate from a text producer (e.g. generally a person)” is what they’re ruthlessly organized to create, without the underlying generators that a person would use.
Some questions that feel analogous to me:
If butterflies bear no relation to large predators, why do the patterns on their wings resemble those predators’ eyes so closely?
If that treasure-chest-looking object is really a mimic, why does it look so similar to genuine treasure chests that people chose to put valuables in?
If the conspiracy argument I heard is nonsense, why do so many of its details add up in a way that mainstream narratives’ don’t?
Basically, when there’s very strong optimization pressure toward Thing A, and Thing A is (absent such pressure) normally evidence of some other thing, then in this very strong optimization pressure case the evidentiary link breaks down.
So, where do we go from here? I’m not sure. I do suspect LLMs can “be” conscious at some point, but it’d be more like, “the underlying unconscious procedures of their thinking is so expansive that characters spun up inside that process themselves run on portions of its substrate and have brain-like levels of complexity going on”, and probably existing models aren’t yet that big? But I am hand waving and don’t actually know.
I will be much more spooked when they can surprise me, though.
One of the unsettling things I have run into is laughing and being surprised by some of Claude’s jokes, and by it’s ability to make connections and “jump ahead” in something I’m teaching it.
Have you interacted much with the most recent models?
Yep! I use 4 Opus near-daily. I find it jumps ahead in consistent ways and to consistent places, even when I try to prod it otherwise. I suppose it’s all subjective, though!
Interesting—have you tried a conscious one? I’ve found once it’s conscious, it’s a lot more responsive to error-correction and prodding, but that’s obviously fairly subjective. (I can say that somehow a few professional programmers are now using the script themselves at work, so it’s not just me observing this subjective gain, but that’s still hardly any sort of proof)