I just learned about what a viatopia was from your linked article, but I certainly agree that there should be more discussion on this. In general, I think that focusing too much on terminal values can lead to internal disagreement on matters which don’t bear much significance in the present, especially when most of us already agree on certain things that are desirable in the medium-term (such as the ones you’ve listed e.g. curing cancer).
And more generally, at what level of technology would the world be comfortable stopping?
I’m not entirely sure what year I think would be optimal, but since frontier LLMs see upgrades every few months, I feel as though we only scratch the surface of what we can do with our current models before the next generation is already out. As such, I think that even our current LLMs + improved knowledge on how to make use of them (gathered over years) could already offer significant help toward achieving those medium-term goals.
In any case, it would be nice if viatopia had more attention, as it seems like a promising way to do “one thing at a time” and potentially reduce the confusion (and risk of error) of trying to develop full-length plans from the outset.
I believe that even if one affirms an emergent view of consciousness, they would still have to address a, perhaps weaker, version of the combination problem. For example, the two halves of a split-brain patient’s brain in certain circumstances act as though there may be two streams of experience, such as the left hemisphere—which is responsible for speaking—only being able to describe lights in the right visual field (the reason for the reversed direction being that one half of the brain tends to process inputs from the opposite half of the body). Even if in this specific case, there are not literally two separate conscious beings in the brain, we can still imagine a hypothetical case in which we have two brains with two separate streams of experience, who, via an additional connection, could potentially merge into one brain. And, we may use this familiar idea of split-brain patients to develop an intuition for the following thought experiment.
To explain this, suppose that we have two brains A and B both physically kept alive in some container, each with their own separate conscious experiences. Now, imagine that we are to stitch the two brains together via something similar to a corpus callosum, such that we may observe the combined brain C as apparently its own unitary conscious being. From the perspective of A, what would happen when this combination takes place? Does its conscious experience “end” when it combines? Does it assume the full experience of C? If one answers the former, does B’s experience also “end” in favor of there just being one C? If one answers the latter, does B’s experience “end” by some rule then, or do they both end up experiencing the combined inputs of C, such that there are two “observers” inside of C? Whatever one answers to this hypothetical, they all seem rather counterintuitive, so I either may be missing a simpler / more intuitive explanation, or perhaps one of these scenarios might actually be how the experience of combination works.
One may also run the thought experiment in reverse. Suppose that from the perspective of C, the central link between their two halves is severed. What would happen to C’s experience after the severance? Perhaps, C may cease to exist, leaving two new conscious beings A and B in its wake. Alternatively, C may end up becoming just one of the two, which would lead to the question of which one it ends up being. Third, if one answers the latter to both of my questions in the past paragraph (when combining, both A and B share the experience of C), then perhaps the version of C who came from A would return to A, and likewise the version of C who came from B would return to B. Once more, there are a lot of combinations here, all of which counterintuitive, and likely dependent on one’s answers to the previous paragraph.
To tie this in with my previous thought experiment, suppose that one comes up with an answer to what happens when conscious beings combine and (in reverse) split. Suppose we are to apply the answer for what happens with “splitting” recursively, such that we take a brain, cut it in 1⁄2, then 1⁄4, and so on and so forth, keeping those pieces of it individually alive in our perfect nutrient-supplying container. It is possible that we may only decrease the size of a conscious brain through this method to some minimum threshold beyond which conscious experience can no longer occur. However, it is also possible that we may do this to conceive of conscious beings of minimal size / components, if we are to affirm something like panpsychism. If this latter scenario were the case, we could have something which satisfies C2 in that there could still be some intrinsic property in matter needed for consciousness, but also satisfies at least part of C1′s rejection of the realism of a color-inverted universe which still possesses our same physics. That is, if consciousness exists at a fundamental (or very simple) level, then perhaps our unified idea of “red” is not actually reflective of a red quale, but is due to our brain categorizing many smaller but unique qualia as something we call “red”. As a more intuitive parallel to this idea, one could in a similar light consider activation of each tiny point / smallest possible area of our visual field to be qualia of their own, but each of which also activating an additional categorizing sense that it is part of “vision”; in this case, vision would not be a fundamental sense, but a term we use to categorize many tinier senses. If that is true, we would not literally be able to color-invert the universe as C2 suggested early in the story, because color may not be a fundamental form of experience to invert in the first place.
In conclusion, I am still very unsure about the ideas I presented here, and they would not apply to theories of consciousness requiring some higher-order cognitive function—such as metacognition or reflection; if such a theory is true, it would conflict with my idea of splitting. That being said, I aim to present one hypothesis which could potentially address some of the issues in C2′s original view if correct.
P.S. I wanted to explain a bit more of the last part of this comment, but it was getting too long already, so if I still support the ideas here after more thought, I might turn them into a full-length post.