For the 2023 review, I’d give it a +9, and perhaps even higher here, mostly due to the well-presented way that the review was structured, and most importantly has for all intents and purposes dissolved a lot of confusions around vexing questions, and the stuff it gets wrong is patchable such that the theory broadly works.
It’s an excellent example of taking a confused question and dissolving it, and I’ll probably refer to this in the future.
Now onto the review itself:
I broadly agree with Anil Seth’s framing of the situation on consciousness, but unlike him, I think it makes sense to say that emulations are conscious, though very weirdly even under his framework, because I think the self-modeling principle is actually pretty universal, and importantly I believe that if we accepted that both consciousness is broadly generated from the need to control a variable of interest by using a model, then I think that the question of whether humans and AI/emulations are conscious do not have independent answers, in that if one of the entities are conscious, then both are conscious, or neither are.
I’d weakly guess both, but I’m also open to a neither answer.
More importantly, I think the need to interact with the outside world can also go beyond the need to stay alive, but yes staying alive is one of the most important goals, and I think instrumental convergence is a big reason for why this can happen.
The most valuable things from this post are:
Optimizing for truth and optimizing for having utility are different, and for the purpose of being alive, you don’t really need to know whether there is such a thing as redness out in the world to recognize that it is useful for utility purposes, and a lot of problems stem from not disentangling truth and utility goals enough. Another way to say it is that internal truth in the brain can be arbitrarily different from external truth.
It focuses on what our consciousness, if it exists is doing, rather than philosophically speculating on the definition of consciousness, as you get way more information that grounds your models that way.
It helps remind ourselves that people are different, and that people have very different experiences of their own generative model, and while they do share commonalities due to living in the same universe with the same laws of physics, people are in fact different, and you can’t always generalize from one person to another.
It’s helpful in dealing with a strain of psychological unity of humankind that was popular on LW.
Quote below:
For one, the decoupling of conscious experience from deterministic external causes implies that there’s truly no such thing as a “universal experience”. Our experiences are shared by virtue of being born with similar brains wired to similar senses and observing a similar world of things and people, but each of us infers a generative model all of our own. For every single perception mentioned in Being You it also notes the condition of having a different one, from color blindness to somatoparaphrenia — the experience that one of your limbs belongs to someone else. The typical mind fallacy goes much deeper than mere differences in politics or abstract beliefs.
4. It points out that the idea of free will is useful for utility purposes, even if it’s not true in some cases, and is another divergence between goals, so it’s useful to not mix these up.
That said, at least for Turing-computable functions, there is a known way to actually do counterfactuals and get sensible results even in a case where there are in fact deterministic laws of physics:
For many people, the aspect of selfhood they cling to most tightly is their volition, the feeling of being the originator and director of their own actions. And if philosophically inclined, they may worry about reconciling this volition with a deterministic universe. Are you truly exercising free will or merely following the laws of physics?
This question betrays that same dualistic map-territory confusion that asks how the material redness of a strawberry could cause the phenomenological redness in your mind. Redness, free will, belief in deterministic physics — these are all features of your generative model. There is no “spooky free will” that violates the laws of physics in our common model of them, but the experience of free will certainly exists and is informative.
Imagine that you are making a cup of tea. When did it feel like you exercised free will? Likely more so at the start of the process, when you observed that all tea-making tools are available to you and contemplated alternatives like coffee or wine. Once you’re deep in the process of making the cup the subsequent actions feel less volitional, executed on autopilot if making tea is a regular habit of yours. What is particular about that moment of initiation?
One particularity is the perception that you are able to predict and control many degrees of freedom: observe and move several objects around, react in complex ways to setbacks, etc. This separates free will from actions that feel “forced” by the configuration of the world outside, like slipping on a wet surface, and reinforces the sensation that free will comes from within.
The experience of volition is also a useful flag for guiding future behavior. If the universe were arranged again in the same exact configuration as when you made tea you will always end up making tea. But the universe (in particular, the state of your brain) never repeats. The feeling of “I could have done otherwise” is the experience of paying attention to the consequences of your action so that in a subjectively similar but not perfectly identical situation you could act differently if the consequences were not as you predicted. If the tea didn’t satisfy as expected, the experience of free will you had when you made it shall guide you the next day to the cold beer you should have drank instead.
Overall, this deserves to be in the LW canon, if only because it takes a confusing set of problems and mostly answers them correctly, and dissolves the questions in some cases rather than solving it, and it also doesn’t use much value-laden language, which is necessary to counteract conflationary alliances.
This would be at the very least an excellent example of a productive mistake, but I don’t think it’s mistaken in a broad theory sense, but rather needs some patching up with other details.
For the 2023 review, I’d give it a +9, and perhaps even higher here, mostly due to the well-presented way that the review was structured, and most importantly has for all intents and purposes dissolved a lot of confusions around vexing questions, and the stuff it gets wrong is patchable such that the theory broadly works.
It’s an excellent example of taking a confused question and dissolving it, and I’ll probably refer to this in the future.
Now onto the review itself:
I broadly agree with Anil Seth’s framing of the situation on consciousness, but unlike him, I think it makes sense to say that emulations are conscious, though very weirdly even under his framework, because I think the self-modeling principle is actually pretty universal, and importantly I believe that if we accepted that both consciousness is broadly generated from the need to control a variable of interest by using a model, then I think that the question of whether humans and AI/emulations are conscious do not have independent answers, in that if one of the entities are conscious, then both are conscious, or neither are.
I’d weakly guess both, but I’m also open to a neither answer.
More here:
https://www.lesswrong.com/posts/Dx9LoqsEh3gHNJMDk/fixing-the-good-regulator-theorem#Making_The_Notion_Of__Model__A_Lot_Less_Silly
My answer on what consciousness likely is, with helpful comments below:
https://www.lesswrong.com/posts/yoAhc7ZhQZfGqrzif/what-are-the-actual-arguments-in-favor-of-computationalism#KTWgPbomupmwE2TFb
https://www.lesswrong.com/posts/TkahaFu3kb6NhZRue/quick-general-thoughts-on-suffering-and-consciousness#zADHLLzykE5hqdgnY
https://www.lesswrong.com/posts/JYsSbtGd2MoGbHdat/book-review-being-you-by-anil-seth#sSjKjjdADWT7feSTc
https://www.lesswrong.com/posts/TkahaFu3kb6NhZRue/quick-general-thoughts-on-suffering-and-consciousness#FaMEMcpa6mXTybarG
https://www.lesswrong.com/posts/TkahaFu3kb6NhZRue/quick-general-thoughts-on-suffering-and-consciousness#WEmbycP2ppDjuHAH2
More importantly, I think the need to interact with the outside world can also go beyond the need to stay alive, but yes staying alive is one of the most important goals, and I think instrumental convergence is a big reason for why this can happen.
The most valuable things from this post are:
Optimizing for truth and optimizing for having utility are different, and for the purpose of being alive, you don’t really need to know whether there is such a thing as redness out in the world to recognize that it is useful for utility purposes, and a lot of problems stem from not disentangling truth and utility goals enough. Another way to say it is that internal truth in the brain can be arbitrarily different from external truth.
It focuses on what our consciousness, if it exists is doing, rather than philosophically speculating on the definition of consciousness, as you get way more information that grounds your models that way.
It helps remind ourselves that people are different, and that people have very different experiences of their own generative model, and while they do share commonalities due to living in the same universe with the same laws of physics, people are in fact different, and you can’t always generalize from one person to another.
It’s helpful in dealing with a strain of psychological unity of humankind that was popular on LW.
Quote below:
4. It points out that the idea of free will is useful for utility purposes, even if it’s not true in some cases, and is another divergence between goals, so it’s useful to not mix these up.
That said, at least for Turing-computable functions, there is a known way to actually do counterfactuals and get sensible results even in a case where there are in fact deterministic laws of physics:
Link below:
https://www.lesswrong.com/posts/mZy6AMgCw9CPjNCoK/computational-model-causal-diagrams-with-symmetry#Why_would_we_want_to_do_this_
Quote below:
Overall, this deserves to be in the LW canon, if only because it takes a confusing set of problems and mostly answers them correctly, and dissolves the questions in some cases rather than solving it, and it also doesn’t use much value-laden language, which is necessary to counteract conflationary alliances.
This would be at the very least an excellent example of a productive mistake, but I don’t think it’s mistaken in a broad theory sense, but rather needs some patching up with other details.