I fully agree that something like persistence/[continued existence in ~roughly the same shape] is the most natural/appropriate/joint-carving way to think about whatever-natural-selection-is-selecting-for in its full generality. (At least that’s the best concept I know at the moment.)
(Although there is still some sloppiness in what does it mean for a thing at time t0 to be “the same” as some other thing at time t1.)
This view is not entirely novel, see e.g., Bouchard’s PhD thesis (from 2004) or the SEP entry on “Fitness” (ctrl+F “persistence”).
I also agree that [humans are]/[humanity is] obviously massively successful on that criterion.
I’m very uncertain as to what implications this has for AI alignment.
I fully agree that something like persistence/[continued existence in ~roughly the same shape] is the most natural/appropriate/joint-carving way to think about whatever-natural-selection-is-selecting-for in its full generality. (At least that’s the best concept I know at the moment.)
(Although there is still some sloppiness in what does it mean for a thing at time t0 to be “the same” as some other thing at time t1.)
This view is not entirely novel, see e.g., Bouchard’s PhD thesis (from 2004) or the SEP entry on “Fitness” (ctrl+F “persistence”).
I also agree that [humans are]/[humanity is] obviously massively successful on that criterion.
I’m very uncertain as to what implications this has for AI alignment.