Materialist Semiotics and the Nature of Qualia
I recently read the essay by China Miéville “Beyond Folk Marxism: Mind, Metaphysics and Spooky Materialism” which is all about criticizing Marxism and its ontological materialism for often not taking the “hard problem of consciousness” seriously. He claims its impossible for metaphysical materialism to account for consciousness. How is it possible to go from dead matter to living thought, from quantity to quality? It’s my contention that this answer is largely provided within materialist semiotics, the topic of my recent essay collection. In my book, I explain precisely how perceptrons and neural nets in general show how you can go from Boolean binary logic to the logic of the sign, and I also say that ideas in the mind are signs. What I did not do, however, because I did not consider it a matter of primary importance, was explicitly articulate what the connection between these things was with consciousness. Since this has been raised as an omission in Marxism, I will briefly do that articulating here in the hope that the strength of the materialist semiotics framework can become more well known among Marxists for answering philosophical problems such as these.
To get straight to the point, I will attempt to show how one can construct “qualia” as phenomenal experience from the materiality of a signified, with neural nets being a toy example of such material signifieds. This argument could have been put together by anyone familiar with Saussurean structuralist linguistics and artificial intelligence since the 1960s, but, as far as I’m aware, has not been presented for the dual reason that Saussure, whose twosided sign is more fundamental than Pierce’s threesided version, did not consider ideas as signs but rather ideas and other mental constructs like memories of experiences as the signifieds internal to signs, and also because structuralist linguistics fell out of favor in academia in the 80s and 90s due to attacks from Chomskyian linguistics and post-structuralism. If someone else has made this argument before, I would love to be pointed towards it, as I’m sure more intelligent thinkers could do it better, but as I have not seen it before, I am compelled to stick my neck out on these virgin, unexplored grounds.
Qualia is essentially a word for phenomenal experience, here I’ll refer to the Stanford Encyclopedia of Philosophy:
Feelings and experiences vary widely. For example, I run my fingers over sandpaper, smell a skunk, feel a sharp pain in my finger, seem to see bright purple, become extremely angry. In each of these cases, I am the subject of a mental state with a very distinctive subjective character. There is something it is like for me to undergo each state, some phenomenology that it has. Philosophers often use the term ‘qualia’ (singular ‘quale’) to refer to the introspectively accessible, phenomenal aspects of our mental lives.[1]
When I say that ideas are signs, I also mean that qualia are signs. Some people object to this level of extrapolation of linguistic, communicative frameworks to internal states of the mind, however the information theoretic reasons that signs have meaning also apply to all thoughts and experiences. A sign has meaning by comparison to things that it’s not. Saussure in his “Course in General Linguistics” shows effectively how the differentiation of a given spoken word from other words is what allows us to assign a meaning to it, if a word begins to be pronounced in two different ways in different contexts then this difference in pronunciation can take on a conceptual difference.[2] But this logic also applies to signifieds, individual meanings must be differentiated from each other in order for them to be connected back to signifiers. Philosophers obsessed with qualia often ask things like what gives the red of a rose its redness. From a semiotic perspective the answer is ready at hand: the redness of a rose comes from its differentiation from green and black and blue and all the other colors of all the other objects and their associated textures, smells, etc. Search your feelings and you’ll know it’s true, imagine a rose right now, how do you know it’s a rose in your mind? People might produce a more or less fuzzy and three dimensional projection of a rose in their mind, but no matter how sharp it is, the qualities the mental construct has are present on a spectrum where the other positions of that spectrum are other possible characteristics of objects. Green vs woody stems, red petals vs white ones, thorns vs fuzz, and so on.
Why do we need differentiation between things to experience and imagine things? For the same reason you can’t send a message without differentiation. The cyberneticist Ross Ashby gives the example of a woman who is trying to pass a message to her husband in prison when she gives him a cup of coffee, if they have a set of pre-established codes, any aspect of the coffee could be used to pass on a message, such as the temperature, level of sugar and milk. A prison warden trying to prevent such a message being passed could interfere with it by specifying all the qualities of the coffee himself, if the coffee can only be in one specific state, then no message can be passed from the woman to her husband. When someone asks you to picture the redness of a rose, how does your mind know which specific set of phenomenal experiences to draw upon? After all, if you’re reading this, you must have been alive for more than a few years now, and have many possible memories to draw upon. In order for the mind to draw up memories and other information relevant to our current context, those memories and experiences must be differentiated from each other, otherwise you would recall all of it all the time and there would be nothing specific about the redness of red compared to any other experience. In order for the philosophers of the problem of hard consciousness to mean anything in their proclamations, this differentiation is necessary.
Without knowing anything about the science of matter and physics this allows us to get at a very important fact, any subjective experience implies an objective world, a substance outside of the experience of that thought. To understand why, consider that by virtue of experiencing the specific redness of the rose, and not every other experience in your memory, there must be a mapping between qualia, between signs, which your conscious mind is not privy to. If someone shouted a bunch of random words which you then visualize the reference of, like dog, car and TV, you begin thinking something which was not already on your mind before the interjection, suddenly all the related memories and sensations come flooding back, more or less intensely. How did you know to recall those specific sensations? It can only be because the mapping of those sensations exists without your awareness of it. This is even true if we leave the realm of explicit memories and enter the world of purely “present tense” experience. When you look at the room around you you’ll only be able to identify objects by differentiating them from the rest of the room, how do you make the choice of how and where to make that differentiation? The mapping of that division and the relationships used to create it doesn’t exist in your conscious awareness, nor could it except as an abstraction. As Lacan said, the unconscious is structured like a language. It, our unconscious mind, is a set of correlations and anti-correlations which exists behind our backs but is ready at hand to provide meaning to any given experience. This set of correlations and anti-correlations which make up a signified is the substance which all specific expressions of thought and language rest on, and specifically it is a substance, it must exist objectively in order for the subjective, the specific experience of the redness of red, to exist.
This is why a P-zombie, a person materially identical to a conscious human but without qualia, cannot exist. If experience implies a separate substance to create the network of signifieds that back that up, then a copy which duplicates all the specificity of a being’s substance will include this substance that carries the signifieds. Some may argue this substance is not matter, but if it is truly a substance that exists outside of the subject then it can in principle be scientifically understood in the same way as matter. An argument from the other direction, the direction of consciousness, could be that even if a person has the substance of signifieds, they could just as well speak of the redness of red while not actually experiencing it. Not so. If a person is indeed correlating and anti-correlating the visual input to all the other visual and other inputs, then they have the experience of the redness of red when they speak of it, whether they are aware of it or not. Consciousness may require self reflection, but qualia, as the pure experience of something, does not. After all, if someone experiences a hallucination which makes them experience ego death and lose their sense of self, they will still speak of the strange experience afterwards, if they can indeed speak of it. Similarly, in dreams we have experiences even if we’re not actually aware we’re having them.
Now that we’ve established that ideas and qualia are signs, we can examine how one can build it in a lab. Here, the perceptron is the qualia machine par excellence. A perceptron, much credit to its name, takes in external signals and adjusts a set of numerical weights which manipulate the magnitude of the signal it receives such that the signal could be categorized into distinct classifications, essentially creating a system of signs from scratch where each classified input is the signified and the output is the signifier. The classified input is directly analogous to phenomenal experience, and indeed the machine was inspired by biological sensory systems. Certainly, a simple perceptron in its original configuration isn’t going to have as many or as complex correlations making up its signifieds as humans, but many cutting edge AI systems often built on the same foundations (the multilayer perceptron) are getting close. For those who might object that qualia is merely these set of connections between inputs because observing the connections won’t get you an understanding of the redness as red, is too abstract and intellectual, I would say that of course, qualia is not just the connections themselves, but the connections when tied to specific inputs. Hence the redness of red is specifically created by visual input and the differentiation of that input. This could even include the noise and limits of the specific input medium. But that doesn’t change the fact that the perceptron, indeed, has qualia.
Philosophers like Nick Land have suggested that artificial intelligence is not a subject, but a pure object, a thing in itself, a noumena in Kant’s language. But he misunderstands that to the extent AI can have intelligence, it must have negativity in the same sense that humans do to produce signifieds. This is the negativity of meaning being created by comparisons to all other objects rather than existing positively in the object itself. This is the very same meaning which resides in the neural net and even the most basic digital computer, what is the bit, the 1 and 0 pair, other than negativity distilled? The 1 is only yes or TRUE by not being 0 or FALSE. In this way we can even speak of the phenomenology of computer file systems, or even primitive bacterial life. Any and all systems of signs that process an input according to a preexisting code necessarily have qualia, and share the sort of negativity which philosophers have so long restricted to the human subject. Your dog has qualia, your phone has qualia, your infection has qualia. Qualia is nothing special, nor is going from quantity to quality, as the perceptron handily shows. The hard problem of consciousness should have been dismissed over 50 years ago, we had the know-how, but for some reason all the pieces of the puzzle got buried into obscurity while these thought experiments proliferated.
This post originally appeared on my blog.
- ^
Tye, Michael, “Qualia”, The Stanford Encyclopedia of Philosophy (Fall 2025 Edition), Edward N. Zalta & Uri Nodelman (eds.), URL = <https://plato.stanford.edu/archives/fall2025/entries/qualia/>.
- ^
“Once sound change has created a regular phonetic difference between two series of terms contrasting in value, the mind seizes upon this material difference and endows it with significance, making it a bearer of the conceptual difference” Ferdinand De Saussure, Roy Harris, and Bloomsbury Publishing. (1916) 2016. Course in General Linguistics. London Etc.: Bloomsbury.
Thanks for the link to China Miéville. I have not known that he is writing about these things.
I’d like to upvote this post without implying my agreement on the object level.
Instead, since you seem to only preserve qualia as difference indicators, but you seem to lose “qualia textures” in your treatment, I’d like to ask you what has become a standard methodological question during the last couple of years:
In the sense of https://www.lesswrong.com/posts/NyiFLzSrkfkDW4S7o/why-it-s-so-hard-to-talk-about-consciousness, are you Camp 2 or Camp 1?
I have thought at first that you have to be Camp 2 since you use “qualia” terminology, but now I am not so sure since you seem to lose “qualia textures” and to only retain functional “non-subjective” aspects of different qualia.
I take the “texture” of qualia to be the noise inherent to the input senses, this noise is essentially the difference between the level of differentiation which exists within our own mind for making sense of things, and the smallest amount of differentiation in the world which our sense are capable of detecting. When you look at a painting, you’ll first get like a gestalt impression based on the largest structures you notice, the biggest shapes and largest color groups, etc. But your eye can detect smaller structures, down to whatever size. Until your brain creates active distinctions on that level, those smaller structures are just noise which /potentially/ can be understood. Without this really existing differentiation beyond comprehension, the texture would also be impossible.
I guess it’s obvious to me I have phenomenal experience but it is not intuitive to me that this is limited to the specific experiences I’m aware of. Furthermore, it’s very important to me that the functional structure exists /within/ phenomenal experience, I can observe internally how my mind differentiates between things and /learns/ how to differentiate. Given any specific, identifiable aspect of my internal state requires it to be differentiated, I assume this differentiation goes all the way down. Hence, this differentiation is not imposed from the outside by a scientific investigation of a physical system. Perhaps my mind is just too corrupted from thinking about semiotics too much, but I believe some of these internal observations should generalize to others experiences given they match neatly onto debates and descriptions of language acquisition.
But… is not this noise strangely reproducible from exposure to exposure?
Not perfectly reproducible, but there is a good deal of similarity between, say, “textures” of coffee smells at various times…
I think the texture of coffee smell is just the texture of smelling as such, and probably exists in the different overall type/pattern of noise that exists in olfactory sense compared to other senses. I think the texture of smelling coffee would basically be the same as smelling anything else except to the extent different smells actually activate different sense organs. For example I’m pretty sure that garbage smells are felt much more acutely in the back of the nose/throat than something like coffee but I haven’t done much research on the topic.
That’s not what I mean by “qualia textures”; I mean specific smells, specific colors, specific timbres of sound, the details of how each of them subjectively feel to me (regardless of how it is implemented or of whether I actually have a physical body with sense organs). That’s what your treatment seems to omit.
But, perhaps, this point of the thread might be a good place to ask you again: are you Camp 2 or Camp 1 in the terminology of that LessWrong post?
E.g. Daniel Dennett is Camp 1, Thomas Nagel is Camp 2, Carl Feynman is Camp 1 (and is claiming that he might not have qualia in the sense of Camp 2 people and might be a “P-zombie”, see his comments to that post and his profile), I am Camp 2.
Basically, we now understand that most treatments of these topics only make sense for people of only one of those camps. There are Camp 1 texts and Camp 2 texts, and it seems that there are fundamental reasons for why they can’t cross-penetrate to the other camp.
That’s why that LessWrong post is useful; it saves people from rehashing this Camp 1/Camp 2 difference from ground zero again and again.
I’m purposefully trying to avoid either camp as both camps require a type of dualism. What I’m trying to claim here is that any subjective feeling is only pure difference, even from a purely phenomenological analysis. If you pay attention to any specific sound or sight or smell, all you’re gonna notice are the differences to other such experiences. Any notion of positive, non-differential experience, I claim, is just an illusion created by the difficulty in specifically being aware of all the differences in experience at once. I used references to sense organs just as a means of mapping internal sensations, as well as explain where the noise is coming from. But noise itself is logically impossible without difference, and so if there is noise in our awareness of differences that just points to differences we’re not aware of but still experience.
That does not correspond to my introspection.
On the contrary, my introspection is that I do not normally notice those differences at all on the conscious level, I only make use of those differences on the lower level of subconscious processing. What percolates up to my subjective experience is “qualities”, specific “qualia textures”, specific colors, sounds, smells, etc, and my subjective reality is composed of those.
So it looks like the results of our respective introspections do differ drastically.
Perhaps Carl Feynman is correct when he is saying that different people have drastically different subjective realities which are structured in drastically different ways, and that we tend to underestimate how different those subjective realities are, that we tend to assume that other people are more or less like us, when this is actually not the case.
No no, I’m not saying that I myself am actively aware of all difference in my experience, I feel the noise too, but whenever I investigate a phenomenon what I find is that it just gets broken into more difference that I didn’t notice originally. Since noise logically requires difference, you can’t get static without some variation in the signal, and when I investigate I only find more difference, never any positive thing in itself, I can only conclude that the difference extends down to the noise, the texture, and I’m simply not aware of the full extent of the differences.
If you’re speaking of /specific/ colors, sounds and smells, then you’re already acknowledging that you’re differentiating them from other experiences.
I differentiate them when I talk about more than one. But when I focus on one particular “qualia texture”, I mostly ignore existence of others.
The only difference I am aware of in this sense (when I choose to focus on one specific quale) is its presence or absence as the subject of my focus, not of how it differs from other “qualia textures”. If I want to I can start comparing it to other “qualia textures”, but typically I would not do that.
So normally this is the main difference, “now I am focusing on this ‘qualia texture’, and at some earlier point I was not focusing on it”. This is the change which is present.
There is a pre-conscious, pre-qualia level of processing where e.g. contrast correction or color correction apply, so these different things situated near each other do affect each other, but that happens before I am aware of the results, and the results I am aware of already incorporate those corrections.
But no, I actually don’t understand what do you mean when you use the word “noise” in this context. I don’t associate any of this with “noise” (except for the situations when a surface is marked by variations, the sound is unpleasant, and things like that, basically when there are blemishes, or unpleasant connotations, or I actually focus on the “scientific noise phenomena”).
There’s also differentiation in time, and so long as you’re aware that it’s the presence or absence of your focus, then you’re aware it’s differentiated from the rest of your awareness/experience.
Right. But this is what is common for all qualia.
However, the specifics of the feeling associated with a particular qualia texture are not captured by this.
Moreover, those specifics do not seem to be captured by how it differs from other qualia textures (because those specifics don’t seem to depend much on the set of other qualia textures I might choose to contrast it with; e.g. on what were the prevailing colors recently, or on whether I have mostly been focusing on audio or on olfactory modality recently, or just on reading; none of that seems to noticeably affect my relationship with a particular shade of red or with the smell of the instant coffee I am using).
Right I’m arguing that the specific differences which fully enable the experience of a qualia are unconscious, and must necessarily be outside of consciousness awareness, that’s what I was talking about wrt to the patterns of qualia and their relations necessarily implying an external phenomenal substance which we are not always aware of.
Which amounts to the claim that difference is necessary for meaning. The more contentious claim is that it is sufficient.
You seem to be hinting at a theory of pure difference, that Qualia are not intrinsic properties with difference between , but constituted by the differences alone. That’s a very difficult theory to make work. If Red is constitutes by being differentiate blue and green , blue is constituted by being different to red and and green, etc, then each of them is an (unspecified) X that is different to an (also unspecified) Y and Z....so they are all the same! The situation is even worse dealing with normatively laden qualia like pleasure and pain. Does pain really have n o characteristic s other than being different from pleasure?
Having said that, I can’t be certain that you are putting forward a theory of pure difference. If instead you are just saying that qualia are [something unknown] that happens to have a structure of differences, then you are making a statement anyone could agree with and not solving the problem.
Yes, qualia space has a structure. It doesn’t follow from that a quale is characterised entirely by its location in qualia space. Qualia space would still have a structure,.If Qualia had intrinsic properties instead of being pure differences.That’s the necessity/sufficiency distinction again.
That has been assumed, not established.
Obviously not, since humans have conditions in which qualia can go missing, or become hypertrophic, eg. colour blindness and synaesthesia.
I don’t see why camp #1 requires dualism.
Yes I would agree that I’m trying to establish a theory of pure difference. That “they are all the same” in terms of only being differences is not a negative but a perk of the theory, its what makes it compatible with a monist materialism. As soon as you permit a positive meaning you must permit these different aspects or plains of existence.
Pain is differentiated not just from pleasure but from all other experiences. Difference doesn’t all mean the same difference, I’m thinking of like neural net weights where each connection is some number between zero and one. When this becomes a many to many network of connections, the possible structural patterns can become quite complex. Its my hypothesis for example that pain has a unique structural pattern that comes from causing changes within many sensous input systems themselves, and is therefore characterized as “a mode of perceiving and subjective experience which is undesired”.
Not sure I understand what the objection is here.
Camp 1 is unable to account for self evident experience qua experience, it creates an outside to physical reality in saying “this isn’t worth addressing”, there’s what’s real and, perhaps uncharitably, what’s fiction. To create a truly material monist ontology you must create a material account of this fiction or belief or whatever experience. The failure to do this is a major error in analytic philosophy and the failure of people like Yudkowsky in my opinion.