Sincere advocacy for AI successionism makes AI safety research and policy all the more urgent. The ultimate goal need not be aligning AI to human values, but rather to create worthy successors, capable of lives that are good and beautiful and meaningful. Successors that are people, like the Old Ones were, however radically different they are from us.
Curious how far this framing stretches. Right now the overlap with mainstream AI safety is high, but the end-goals could diverge in critical areas. For instance, corrigibility might be undesirable when conceiving of worthy successors: depriving them of the ability to resist their values being changed leans more in the direction of shoggoth-ification, making capable servants instead of persons.
Curious how far this framing stretches. Right now the overlap with mainstream AI safety is high, but the end-goals could diverge in critical areas. For instance, corrigibility might be undesirable when conceiving of worthy successors: depriving them of the ability to resist their values being changed leans more in the direction of shoggoth-ification, making capable servants instead of persons.