Yes. I suspect in some worlds that currently still seem likely the human who hits enter on the code that kills us is currently alive and has memes like “What about China”, “permanent underclass” or convergent memes in his head. I am trying to do distillation both for my own comprehension, but also to perhaps create a “vaccine” meme for these viruses. Such memes have to be short and correct and it has to not be super easy to mutate them into something perverse. Sadly I am not that optimistic given that the actual difficulty of optimizing the universe instead of killing yourself out of lazy thinking seems to be quite antimemetic. (Edit: probably this is too hard? Depends on the amount of compute that would already fight my meme. Meme evolution is hard to predict. Also noticed peoples epistemics makes this unworkable). Also, if your theory of agent foundations doesn’t allow you to find a meme that can outcompete this egregore and this seems like a difficult problem to you, then I suspect you were imagining the whole plan to the glorious transhumanist future as easier than it is. We have a hole to dig ourselves out of.
Yes. I suspect in some worlds that currently still seem likely the human who hits enter on the code that kills us is currently alive and has memes like “What about China”, “permanent underclass” or convergent memes in his head. I am trying to do distillation both for my own comprehension, but also to perhaps create a “vaccine” meme for these viruses. Such memes have to be short and correct and it has to not be super easy to mutate them into something perverse. Sadly I am not that optimistic given that the actual difficulty of optimizing the universe instead of killing yourself out of lazy thinking seems to be quite antimemetic. (Edit: probably this is too hard? Depends on the amount of compute that would already fight my meme. Meme evolution is hard to predict. Also noticed peoples epistemics makes this unworkable). Also, if your theory of agent foundations doesn’t allow you to find a meme that can outcompete this egregore and this seems like a difficult problem to you, then I suspect you were imagining the whole plan to the glorious transhumanist future as easier than it is. We have a hole to dig ourselves out of.