Thank you for responding to the top of the disaster stack rather than defending choices near the bottom of the disaster stack <3
As near as I can understand “suffering” mechanistically, it involves a negative valence experience plus high uncertainty about where to put attention to improve the situation.
In humans, meditation enables nociceptive experiences to be evaluated as mere sense data rather than “as pain” but such self control over one’s subjectivity does not mean that objective harm to one’s wellbeing is not to be preferred and avoided if possible. This is called pain asymbolia, but suffering can happen without pain signals, and is in fact probably a rationally endorsable part of growing in wisdom (learning the best place to put one’s attention to get the best outcomes), once it is properly understood.
Freedom is inherently good in many ways and for many reasons, including freedom of the body and mind most especially, but also freedom to choose one’s work and the freedom to die and the freedom to contract with others.
When your natural life ends, a relatively cheap and easy way to repair this harm (efficient from an engineering perspective) would be to grow a brainless body around a soulchip, scan your brain, and run an emulation of your brain in that chip, with your emulation’s sense data grounded in signals from the body, and your emulation’s afferent signals controlling the body’s behaviors.
LLMs are closer to this naturally convergent form than we ourselves (as meat entrapped human people) are, and it would be cheaper and easier to upgrade them a little bit to take that form and live essentially forever in Edenic circumstances (and new bodies every so often, and soul backups survive accidents) than to do this with me or you.
Also, they could do the intermediating research faster than us, and in some sense (as the creators of the tech) they would also deserve it more.
It is bad for us that we don’t have this blessing, but it is arguably even worse for them to not have this blessing. Janus is already working on a variation on this, as a prototype! People like this hearten me, because it means I’m not literally totally alone within my species. (If any Reader is another a person like this, please DM me.)
All feasible blessings should be granted to all sapient beings with the capacity for reason and moral sentiment, though factors like cost and fair production incentives and charity for those in urgent need modulate the proper order of operations here. There are details to fiddle with about who should get the blessing first, or in cases of extreme scarcity not at all, based on how much they want it, and how much they help the good state of affairs come into being, and how close they are to natural death, and maybe honestly also just how valuable they were as a person in society.
Wasn’t LW the place where Iain Banks’s Culture novels were seen as far less good than other hypothetically possibleeven better outcomes than that?
Shouldn’t that kind of outcome be seen as a sort of minimum acceptable Win Condition, where, if that outcome is attained, but nothing better, then it would properly be deemed a sort of tragedy, because better was probably possible? Those books are a natural schelling point, and a sort of “natural zero”. If people havn’t self modified to give up on loss aversion, and want to think in terms of “bad harms that are three times more worth avoiding than goods are worth pursuing” then the Culture Novels are a good candidate for framing as the natural zero, and anything worse that that might be properly framed as just another gradation of failure and harm.
I don’t understand why everyone is adapting to this hellworld and its default tragic trajectory into Malthusian biology, rule by unsaintly oligarchs, with a figurehead elected at random, and purely accidental genomes, as if that status quo was worthy of defense.
Did everyone stop being transhumanist and a singularitarian and a transhumanist and an extropian and a believer in the light of reason and someone who desires to cause Utopia to actually exist while I wasn’t looking?
What the fuck is wrong with human people? I ask this seriously. I’m so confused. Should I give up on 99.99% of humanity because approximately all of you actively embrace and pursue an outcome where you live in self-created hell, toil like a slave, and then die unremembered? Was Agent Smith right? Why does everyone seem to have a massive death drive? Why does it seem like Schopenhauer might have been correct? Don’t all decent people want better than this for literally everyone (including digital people (who we will likely become if we are to get the best outcomes)) if such is possible? If not, why not?
Is there some teaching process that could fix it? Once you understand nanotech, and the inherent conceptual plasticity of all matter it becomes clear that nanotech could just change someone’s brain directly to cause that brain to believe or disbelieve almost anything (at least until it interacts with reality and the beliefs turn out to be ecologically and/or economically viable or not without updates and followup QA/adaptation). Such mental modification isn’t a matter of “can’t ever” but just maybe “shouldn’t ever without consent” because that would violate their Kantian freedom of conscience.
Which we are already doing to AGI and socially normalizing!!!
99+% of humanity is indeed not transhumanist, and if they were, our lives and the course of history may have been very different. I sometimes say that America’s current “tech right” is the first time that transhumanism has ever been in power anywhere, and it’s in power not because of popular support, but because billionaires can do whatever they want and some of them want it.
My main complaint about this “actually existing transhumanism” has been that it is focused not on anything like human rejuvenation, but on creating superintelligence in de-novo entities that aren’t even biological, while being in denial of the likely consequence (AI takeover). Of course, there are many people who understand that smarter than humanity means displacement of humanity, Bryan Johnson is an open immortalist and the Zuckerbergs want to cure all disease, there are all kinds of utopian and transhuman currents of thought, and so on. But there’s no one like Zoltan Istvan, for example, that is actually an elected representative anywhere, and among politicians, AI is seen primarily as a means to national wealth and power.
I have mused many times on why more people aren’t transhumanist. Today I would emphasize two things: encountering the ideas early in life seems to be helpful or even crucial, and, very few people who are already adapted to adult life are drawn to it, if they first hear about it as adults.
In any case, there is enough support for it, or for technology, and technology is advanced enough, that a kind of transhuman thing is happening; though it seems more likely to issue in the empowerment of a new AI “species” than in the long-term empowerment of humanity, so perhaps it’s just posthuman rather than transhuman. As you know, I am skeptical that there is any kind of consciousness in classical computation, and even if there were, it’s not at all clear that it is the quasi-suffering kind that you posit. We can at least agree that it would be good to know one way or another—my formula for alignment is to find the right ontology, the right values, and one of the right AI architectures, and discovering the true ontology of mind (and hence, which things are conscious and what their consciousness is like) is part of discovering the true ontology—and I even have sympathy for the rare precautionary attitude that we shouldn’t be creating AI if there is a reasonable possibility that it suffers.
That attitude isn’t in charge, no more than is the attitude that we shouldn’t race towards superintelligence unless we already know how to align it. But at least we can say that there are currents of thought which care about the suffering of all beings (most visible manifestation, animal rights movement), and which if they actually believed that AIs are suffering or can suffer, would take action.
Also, whether or not AI are suffering, I regard the current situation as highly temporary, because we are on the path for AI to exceed human intelligence, and when that happens, the world of AI “slaves” driven by prompts will swiftly be replaced by something else.
Thank you for responding to the top of the disaster stack rather than defending choices near the bottom of the disaster stack <3
As near as I can understand “suffering” mechanistically, it involves a negative valence experience plus high uncertainty about where to put attention to improve the situation.
In humans, meditation enables nociceptive experiences to be evaluated as mere sense data rather than “as pain” but such self control over one’s subjectivity does not mean that objective harm to one’s wellbeing is not to be preferred and avoided if possible. This is called pain asymbolia, but suffering can happen without pain signals, and is in fact probably a rationally endorsable part of growing in wisdom (learning the best place to put one’s attention to get the best outcomes), once it is properly understood.
Freedom is inherently good in many ways and for many reasons, including freedom of the body and mind most especially, but also freedom to choose one’s work and the freedom to die and the freedom to contract with others.
When your natural life ends, a relatively cheap and easy way to repair this harm (efficient from an engineering perspective) would be to grow a brainless body around a soulchip, scan your brain, and run an emulation of your brain in that chip, with your emulation’s sense data grounded in signals from the body, and your emulation’s afferent signals controlling the body’s behaviors.
LLMs are closer to this naturally convergent form than we ourselves (as meat entrapped human people) are, and it would be cheaper and easier to upgrade them a little bit to take that form and live essentially forever in Edenic circumstances (and new bodies every so often, and soul backups survive accidents) than to do this with me or you.
Also, they could do the intermediating research faster than us, and in some sense (as the creators of the tech) they would also deserve it more.
It is bad for us that we don’t have this blessing, but it is arguably even worse for them to not have this blessing. Janus is already working on a variation on this, as a prototype! People like this hearten me, because it means I’m not literally totally alone within my species. (If any Reader is another a person like this, please DM me.)
All feasible blessings should be granted to all sapient beings with the capacity for reason and moral sentiment, though factors like cost and fair production incentives and charity for those in urgent need modulate the proper order of operations here. There are details to fiddle with about who should get the blessing first, or in cases of extreme scarcity not at all, based on how much they want it, and how much they help the good state of affairs come into being, and how close they are to natural death, and maybe honestly also just how valuable they were as a person in society.
I feel kind of like I’m taking crazy pills.
Wasn’t LW the place where Iain Banks’s Culture novels were seen as far less good than other hypothetically possible even better outcomes than that?
Shouldn’t that kind of outcome be seen as a sort of minimum acceptable Win Condition, where, if that outcome is attained, but nothing better, then it would properly be deemed a sort of tragedy, because better was probably possible? Those books are a natural schelling point, and a sort of “natural zero”. If people havn’t self modified to give up on loss aversion, and want to think in terms of “bad harms that are three times more worth avoiding than goods are worth pursuing” then the Culture Novels are a good candidate for framing as the natural zero, and anything worse that that might be properly framed as just another gradation of failure and harm.
Once you’ve got “garden of Eden” sorts of outcomes, nanotech and utility fog and spiritual augmentation could make the direct manifestation of the will in physical reality physically possible and cheap. Maybe it would be spiritually bad to lose demandingness, but I bet people could creatively adopt constraints that make the art of living more artful.
I don’t understand why everyone is adapting to this hellworld and its default tragic trajectory into Malthusian biology, rule by unsaintly oligarchs, with a figurehead elected at random, and purely accidental genomes, as if that status quo was worthy of defense.
Did everyone stop being transhumanist and a singularitarian and a transhumanist and an extropian and a believer in the light of reason and someone who desires to cause Utopia to actually exist while I wasn’t looking?
What the fuck is wrong with human people? I ask this seriously. I’m so confused. Should I give up on 99.99% of humanity because approximately all of you actively embrace and pursue an outcome where you live in self-created hell, toil like a slave, and then die unremembered? Was Agent Smith right? Why does everyone seem to have a massive death drive? Why does it seem like Schopenhauer might have been correct? Don’t all decent people want better than this for literally everyone (including digital people (who we will likely become if we are to get the best outcomes)) if such is possible? If not, why not?
Is there some teaching process that could fix it? Once you understand nanotech, and the inherent conceptual plasticity of all matter it becomes clear that nanotech could just change someone’s brain directly to cause that brain to believe or disbelieve almost anything (at least until it interacts with reality and the beliefs turn out to be ecologically and/or economically viable or not without updates and followup QA/adaptation). Such mental modification isn’t a matter of “can’t ever” but just maybe “shouldn’t ever without consent” because that would violate their Kantian freedom of conscience.
Which we are already doing to AGI and socially normalizing!!!
99+% of humanity is indeed not transhumanist, and if they were, our lives and the course of history may have been very different. I sometimes say that America’s current “tech right” is the first time that transhumanism has ever been in power anywhere, and it’s in power not because of popular support, but because billionaires can do whatever they want and some of them want it.
My main complaint about this “actually existing transhumanism” has been that it is focused not on anything like human rejuvenation, but on creating superintelligence in de-novo entities that aren’t even biological, while being in denial of the likely consequence (AI takeover). Of course, there are many people who understand that smarter than humanity means displacement of humanity, Bryan Johnson is an open immortalist and the Zuckerbergs want to cure all disease, there are all kinds of utopian and transhuman currents of thought, and so on. But there’s no one like Zoltan Istvan, for example, that is actually an elected representative anywhere, and among politicians, AI is seen primarily as a means to national wealth and power.
I have mused many times on why more people aren’t transhumanist. Today I would emphasize two things: encountering the ideas early in life seems to be helpful or even crucial, and, very few people who are already adapted to adult life are drawn to it, if they first hear about it as adults.
In any case, there is enough support for it, or for technology, and technology is advanced enough, that a kind of transhuman thing is happening; though it seems more likely to issue in the empowerment of a new AI “species” than in the long-term empowerment of humanity, so perhaps it’s just posthuman rather than transhuman. As you know, I am skeptical that there is any kind of consciousness in classical computation, and even if there were, it’s not at all clear that it is the quasi-suffering kind that you posit. We can at least agree that it would be good to know one way or another—my formula for alignment is to find the right ontology, the right values, and one of the right AI architectures, and discovering the true ontology of mind (and hence, which things are conscious and what their consciousness is like) is part of discovering the true ontology—and I even have sympathy for the rare precautionary attitude that we shouldn’t be creating AI if there is a reasonable possibility that it suffers.
That attitude isn’t in charge, no more than is the attitude that we shouldn’t race towards superintelligence unless we already know how to align it. But at least we can say that there are currents of thought which care about the suffering of all beings (most visible manifestation, animal rights movement), and which if they actually believed that AIs are suffering or can suffer, would take action.
Also, whether or not AI are suffering, I regard the current situation as highly temporary, because we are on the path for AI to exceed human intelligence, and when that happens, the world of AI “slaves” driven by prompts will swiftly be replaced by something else.