The intent was to make that taboo permanent, not a “pause”, and it more or less succeeded in that.
I would not be opposed to a society stalled at 2016 level AI/computing that held that level indefinitely. Progress can certainly continue without AGI via e.g human intelligence enhancement or just sending our best and brightest to work directly on our problems instead of on zero-sum marketing or AI efforts.
Um, humans in the Culture have no significant influence on the lightcone (other than maybe as non-agentic “butterfly wings”). The Minds decide what’s going to happen
Humans were still free to leave the Culture, however; not all of the lightcone was given to the AI. Were we to develop aligned ASI, it would be wise to slice off a chunk of the lightcone for humans to work on “on their own.”
I don’t think the Culture is an ideal outcome, either, merely a “good” one that many people would be familiar with. “Uplifting” humans rather than developing replacements for them will likely lead us down a better path, although the moral alignment shift in whatever the uplifting process is might limit its utility.
I would not be opposed to a society stalled at 2016 level AI/computing that held that level indefinitely. Progress can certainly continue without AGI via e.g human intelligence enhancement or just sending our best and brightest to work directly on our problems instead of on zero-sum marketing or AI efforts.
Humans were still free to leave the Culture, however; not all of the lightcone was given to the AI. Were we to develop aligned ASI, it would be wise to slice off a chunk of the lightcone for humans to work on “on their own.”
I don’t think the Culture is an ideal outcome, either, merely a “good” one that many people would be familiar with. “Uplifting” humans rather than developing replacements for them will likely lead us down a better path, although the moral alignment shift in whatever the uplifting process is might limit its utility.