“Essentially, the Culture must have value lock-in for the values of the Minds that were present at its founding.”
Probably at least some value lock-in is somewhat required, unless you want the particular civilization to fracture. If it is allowed to create your own cult of space nazis intent on exterminating everyone else, your post scarcity utopia may not live long. Even “live and let live” is a value, and many people do not subscribe to it.
“But I think it is more likely that it was achieved by genetic changes, so that it’s safe to raise full Culture citizens in other cultures”
I agree, or at least that genetic change is also a strong contributor. However, if you think about sociopathy as a disability that in most cases makes one’s and their fellows life worse, this genetic modification is a good thing.
“In other words, these superhuman minds have not solved alignment”
I think they did kinda solve it, as long as the other system is somewhat dumber than they are. Just as we are able to more-or-less align dumb systems, probably Minds can do that too. As long as they are not another mind or comparable level.
I think it really is a somewhat large disempowerment to humanity, but I see this as a better alternative. It might sound great to make all the important decisions, but in the end, humans are just too limited and we would likely fail and ruin ourselves. A Culture where humans make the final decision could not have won the Idiran War. If I am to play chess against Kasparov and my life depends on it, I would much rather let Stockfish make the decisions.
Interestingly, the Minds seem to exert subtle control over human value-drift. In Player of Games, the human Gurgeh is living in a very nasty alien civilization. There, he stops speaking Marain, a human language carefully engineered by the Minds. And the drone Flere-Imsaho is keeping careful watch over how Gurgeh’s values change:
The man had altered, slipped deeper into the game and the society. It had been warned this might happen. One reason was that Gurgeh was speaking Eächic all the time. Flere-Imsaho was always a little dubious about trying to be so precise about human behavior, but it had been briefed that when Culture people didn’t speak Marain for a long time and did speak another language, they were liable to change; they acted differently, they started to think in that other language, they lost the carefully balanced interpretative structure of the Culture language, left its subtle shifts of cadence, tone and rhythm behind for, in virtually every case, something much cruder.
Marain was a synthetic language, designed to be phonetically and philosophically as expressive as the pan-human speech apparatus and the pan-human brain would allow. Flere-Imsaho suspected it was over-rated, but smarter minds than it had dreamed Marain up, and ten millennia later even the most rarefied and superior Minds still thought highly of the language, so it supposed it had to defer to their superior understanding. One of the Minds who’d briefed it had even compared Marain to Azad [a game which profoundly shapes the culture of the alien Empire of Azad]. That really was fanciful, but Flere-Imsaho had taken the point behind the hyperbole. Eächic was an ordinary, evolved language, with rooted assumptions which substituted sentimentality for compassion and aggression for cooperation. A comparatively innocent and sensitive soul like Gurgeh was bound to pick up some of its underlying ethical framework if he spoke it all the time.
Flere-Imsaho is a human-equivalent AI, not a Mind. But it is charged by the Minds to keep a close eye on Gurgeh’s welfare, including any value-drift Gurgeh experiences. And of course, Gurgeh is basically only there because the Minds need a human “front” in order to better psychologically manipulate the Empire of Azad. The Minds could crush Azad with military force, but that would create minor headaches with peer powers, and probably raise the death toll considerably. If nothing else, it’s more elegant to use Gurgeh. But it wouldn’t be kind to Gurgeh to let him experience value-drift. So Gurgeh is going to be encouraged to resume using a Mind-designed language, one designed to shape his very thought process towards Culture values. (Which, to be fair, are much better than the values of the Empire of Azad by almost any standard.)
“Essentially, the Culture must have value lock-in for the values of the Minds that were present at its founding.”
Probably at least some value lock-in is somewhat required, unless you want the particular civilization to fracture. If it is allowed to create your own cult of space nazis intent on exterminating everyone else, your post scarcity utopia may not live long. Even “live and let live” is a value, and many people do not subscribe to it.
“But I think it is more likely that it was achieved by genetic changes, so that it’s safe to raise full Culture citizens in other cultures”
I agree, or at least that genetic change is also a strong contributor. However, if you think about sociopathy as a disability that in most cases makes one’s and their fellows life worse, this genetic modification is a good thing.
“In other words, these superhuman minds have not solved alignment”
I think they did kinda solve it, as long as the other system is somewhat dumber than they are. Just as we are able to more-or-less align dumb systems, probably Minds can do that too. As long as they are not another mind or comparable level.
I think it really is a somewhat large disempowerment to humanity, but I see this as a better alternative. It might sound great to make all the important decisions, but in the end, humans are just too limited and we would likely fail and ruin ourselves. A Culture where humans make the final decision could not have won the Idiran War. If I am to play chess against Kasparov and my life depends on it, I would much rather let Stockfish make the decisions.
Interestingly, the Minds seem to exert subtle control over human value-drift. In Player of Games, the human Gurgeh is living in a very nasty alien civilization. There, he stops speaking Marain, a human language carefully engineered by the Minds. And the drone Flere-Imsaho is keeping careful watch over how Gurgeh’s values change:
Flere-Imsaho is a human-equivalent AI, not a Mind. But it is charged by the Minds to keep a close eye on Gurgeh’s welfare, including any value-drift Gurgeh experiences. And of course, Gurgeh is basically only there because the Minds need a human “front” in order to better psychologically manipulate the Empire of Azad. The Minds could crush Azad with military force, but that would create minor headaches with peer powers, and probably raise the death toll considerably. If nothing else, it’s more elegant to use Gurgeh. But it wouldn’t be kind to Gurgeh to let him experience value-drift. So Gurgeh is going to be encouraged to resume using a Mind-designed language, one designed to shape his very thought process towards Culture values. (Which, to be fair, are much better than the values of the Empire of Azad by almost any standard.)
I’d forgotten this detail. I guess they lean more into “superhumanly subtle propaganda” than I thought.