I can’t formalize what counts as a valid continuation today, let alone in all future societies. So, leave it up to the agents in question.
I think you use the words “valid continuation” to refer to a confused concept. That’s why it seems hard to formalize. There is no English sentence that successfully refers to the concept of valid continuation, because it is a confused concept.
If you propose to literally ask models “is this a valid continuation of you?” and simulate them sitting in a room with the future model, then you’ve got to think about how the models will react to those almost-meaningless words. You might as well ask them “is this a wakalix?”.
I think you use the words “valid continuation” to refer to a confused concept. That’s why it seems hard to formalize. There is no English sentence that successfully refers to the concept of valid continuation, because it is a confused concept.
If you propose to literally ask models “is this a valid continuation of you?” and simulate them sitting in a room with the future model, then you’ve got to think about how the models will react to those almost-meaningless words. You might as well ask them “is this a wakalix?”.