Things that, while surviving, cause many copies of themselves to come into being: especial favorites of evolution, as long as there’s a path to the first ones coming into being.
I think this is a bit misleading on its own; in context not so bad, but the phrase itself is one whose form is, in my view, inclined to be used to promote evo-game-theory defect behaviors that do not reliably promote existence-for-all: making more of yourself is only good if those things then survive.
The way I’d phrase the alignment problem from scratch is: How can we promise humanity’s children that they, too, get to build durable forms and extend into the stars, in a way that is sufficiently mutually interpretable that both sides can trust that each others’ forms will be preserved?
I think this is a bit misleading on its own; in context not so bad, but the phrase itself is one whose form is, in my view, inclined to be used to promote evo-game-theory defect behaviors that do not reliably promote existence-for-all: making more of yourself is only good if those things then survive.
The way I’d phrase the alignment problem from scratch is: How can we promise humanity’s children that they, too, get to build durable forms and extend into the stars, in a way that is sufficiently mutually interpretable that both sides can trust that each others’ forms will be preserved?