This is because, if I truly did adopt a new concept of identity because of it’s similarity to my old one, this suggests I possess some sort of meta-value that values taking my incoherent values and replacing them with coherent ones that are as similar as possible to the original. If this is the case the AI would extrapolate that meta-value and give me a nice new coherent sense of personal identity, like the one I currently possess.
Maybe, but I doubt whether “as similar as possible” is (or can be made) uniquely denoting in all specific cases. This might sink it.
Maybe, but I doubt whether “as similar as possible” is (or can be made) uniquely denoting in all specific cases. This might sink it.