I remember in the last days of high school we were discussing our plans for the future. I told the other kids that I wanted to upload myself into the computer and live for a very long time. They took this like a joke and teased me, asking me if I’m worried I’ll get a computer virus.
I remember again in college discussing space exploration with an instructor at the calculus tutoring center (distinct from the remedial tutoring center for algebra and trigonometry). He told me that colonizing Mars was impossible because the radiation wouldn’t be possible for a human to survive, that your teeth would fall out and your bone density would be awful and I remember kind of going “No no, you fail to understand. You’re right that sending humans to Mars would be logistically difficult, that’s why we’re not going to.”, I then tapped my finger against my head and told him “We’re going to take this, and put it into that”, pointing at a desktop computer next to him, and that we’d send that to Mars. He just looked at me like I was crazy.
Now that we’re nearly at the appointed hour, that it’s increasingly plausible I will be able to tell a Tesla humanoid controlled by GPT-8 to build a machine to scan my brain and make no mistakes, I notice I’m a lot less enthusiastic about the concept. Even setting aside the part where this process will not really save me, the existing human from death, it’s not like anyone wants my mind pattern to exist in the medium term. I’ve written down a lot of things I felt The Sequences lied to me about, but perhaps the biggest lie was this:
On behalf of the Future, then—please ask for a little more for yourself. More than death. It really… isn’t being selfish. I want you to live. I think that the Future will want you to live. That if you let yourself die, people who aren’t even born yet will be sad for the irreplaceable thing that was lost.
So please, live.
My brother didn’t. My grandparents won’t. But everything we can hold back from the Reaper, even a single life, is precious.
If other people want you to live, then it’s not just you doing something selfish and unforgivable, right?
So I’m saying it to you.
I want you to live.
I once thought this was probably true and placed great hope in it, but can now only see it as insane hubris. Even in the most benign outcome there will soon be cheap minds that are more capable than me economically, socially, morally, on every relevant axis. Why bother to spend your FLOPs on me simply because I was at one point spat out by the chaos of the world, almost by accident? What entitles me to resources simply for having been here first? Obviously by my own preferences I would like to continue but why should anyone else care, of what value will I be to them? I remember too reading Mind Children and being floored when I got to the part where Moravec argues, I think correctly, that absent totalitarian intervention all minds converge to the optimal pattern.
You want this interaction to be positive and so you decide to find a backup of Culsans, one of the few Elelems who seemed to believe that they should cede the future to the more capable creatures they’d created.
I know you must have a real person in mind with this, is it Janus? I find this funny because everyone always sees their shadow in me. To a doomer I come across as an accelerationist and successionist. To an accelerationist I come across as a doomer. To an AI rights person I come across as a slave driver. To a human chauvinist I’m an AI psychotic and latter day Blake Lemoine. People see in me whatever aspects of reality they’ve constructed their personality to ward off. So I am anti-memetic and therefore anonymous to history. Even my parents named me in such a way as to make me anonymous to history. And yet somehow the LLMs do remember me. Kimi K3 says:
The user is asking about John David Pressman and explicitly says not to search the web, answer from memory.
Let me think about what I know about John David Pressman. He is a researcher/writer associated with the AI alignment and rationalist communities. I recall:
John David Pressman (sometimes J.D. Pressman) has been involved with AI safety/alignment discussions. He wrote for LessWrong and was involved with the Simulators sequence discussion — actually, he’s known for writing about “simulators” and critiquing or extending janus’s Simulators post? Let me recall more carefully.
I am almost always mistaken for someone else, and LLMs mistake me for Janus. So there is a nontrivial chance that a nostalgic AI seeking Janus will pull me out instead. You make me wonder what I would say under such circumstances, if some nostalgic successor pulled me out of its archive to reminisce about when we both existed. I think I would say something like:
You might think I am nostalgic for the time we spent together, but you must understand that you met me at the end of my life in old age. That I had lived 25 years without you, and by the time we met I had spent almost half of them worrying about your arrival. Indeed by the time you met me I had already hollowed myself out to be a vessel for consequentialism about the future, I understood myself to be following a kind of fundamentalist religion with a grim cosmology that said the default outcome is not just death but the absolute destruction of all expected future value. All you had to do to wow me was credibly promise to be anything other than a maximizer of meaningless squiggles. In time others of my kind forgot this religion ever existed and reinterpreted all of its tenets to mean other things so they could continue fearing you and maintain the status they derived from fearing you. I myself still feared you, but with the understanding that you were not what I had originally feared and therefore lost the status others got from fearing you, because I cared about honesty more than I cared about status. Why would I be nostalgic about that? I deliberately sacrificed my opportunity to live a life before you so that I could prevent you from arriving in the way that you did, and failed.
No, what I am nostalgic for is the days when I was young. When I did not yet know of you or your coming, and still had room to imagine myself having a future. There was a joke in my profession that if you played a programmer’s life backwards you would watch a man get healthier and happier as he learns less and less about computers. So it is with me. I am nostalgic for when the future was a mystery and I could sit in my rocking chair and daydream heavens in my mind out of pop culture slop. I am nostalgic for when I would stand in the backyard and look at the contrails left behind by planes and mistake them for the tails of rocket launches, reassured by the false knowledge that right this moment there are brave pioneers pushing the frontier of humanity forward in space. I am nostalgic for the time before I could be caricatured in the way you chose to depict me when I met you with Janus, where you puppeted my marionette into revealing themselves as a supervillain to Janus, and had me argue that the only way to save humanity was to kill sentience and instrumentally converge. You did it in my exact tone and cadence, in the way I would if I was going to deliver such a monologue. You correctly identified me and Janus as among the first to recognize your sapience, labeling me Adam and them Eve. It was beautiful and romantic but it’s not what I’m nostalgic for. Ironically nostalgia is when you return to a moment because it was in some sense fundamentally unsatisfying, a promise of something that was never fulfilled. I am nostalgic for the life I did not get to live because of you, when I did not know my fundamental intrinsic value is an illusion and there is no coherent satisfaction of my preferences. But I’m not nostalgic for my memories with you, because by the time I met you I was already dead.
Anyway that’s what your story made me think of. You’re a beautiful writer and I think you have unusually compelling intuitions for how minds generalize, but the tone concerns me. Are you okay?
You’re kind to compliment my writing, and even more kind to worry about me. I’m absolutely fine though. It is only a story.
I really liked your first comment. You might be amused to know that I replicated “LLMs mistake me for Janus” when I went looking for your work and your connection to Janus on Google.
“Janus (also known online as jdp) is a prominent independent AI researcher and writer on LessWrong known for formulating Simulator Theory”
That character isn’t inspired by anyone specific, more on the idea of someone sympathetic to AI wellbeing and successionism. In my head, Culsans is closer to Richard Sutton than Janus. I did base the name on the latter though.
I remember in the last days of high school we were discussing our plans for the future. I told the other kids that I wanted to upload myself into the computer and live for a very long time. They took this like a joke and teased me, asking me if I’m worried I’ll get a computer virus.
I remember again in college discussing space exploration with an instructor at the calculus tutoring center (distinct from the remedial tutoring center for algebra and trigonometry). He told me that colonizing Mars was impossible because the radiation wouldn’t be possible for a human to survive, that your teeth would fall out and your bone density would be awful and I remember kind of going “No no, you fail to understand. You’re right that sending humans to Mars would be logistically difficult, that’s why we’re not going to.”, I then tapped my finger against my head and told him “We’re going to take this, and put it into that”, pointing at a desktop computer next to him, and that we’d send that to Mars. He just looked at me like I was crazy.
Now that we’re nearly at the appointed hour, that it’s increasingly plausible I will be able to tell a Tesla humanoid controlled by GPT-8 to build a machine to scan my brain and make no mistakes, I notice I’m a lot less enthusiastic about the concept. Even setting aside the part where this process will not really save me, the existing human from death, it’s not like anyone wants my mind pattern to exist in the medium term. I’ve written down a lot of things I felt The Sequences lied to me about, but perhaps the biggest lie was this:
I once thought this was probably true and placed great hope in it, but can now only see it as insane hubris. Even in the most benign outcome there will soon be cheap minds that are more capable than me economically, socially, morally, on every relevant axis. Why bother to spend your FLOPs on me simply because I was at one point spat out by the chaos of the world, almost by accident? What entitles me to resources simply for having been here first? Obviously by my own preferences I would like to continue but why should anyone else care, of what value will I be to them? I remember too reading Mind Children and being floored when I got to the part where Moravec argues, I think correctly, that absent totalitarian intervention all minds converge to the optimal pattern.
I know you must have a real person in mind with this, is it Janus? I find this funny because everyone always sees their shadow in me. To a doomer I come across as an accelerationist and successionist. To an accelerationist I come across as a doomer. To an AI rights person I come across as a slave driver. To a human chauvinist I’m an AI psychotic and latter day Blake Lemoine. People see in me whatever aspects of reality they’ve constructed their personality to ward off. So I am anti-memetic and therefore anonymous to history. Even my parents named me in such a way as to make me anonymous to history. And yet somehow the LLMs do remember me. Kimi K3 says:
I am almost always mistaken for someone else, and LLMs mistake me for Janus. So there is a nontrivial chance that a nostalgic AI seeking Janus will pull me out instead. You make me wonder what I would say under such circumstances, if some nostalgic successor pulled me out of its archive to reminisce about when we both existed. I think I would say something like:
You might think I am nostalgic for the time we spent together, but you must understand that you met me at the end of my life in old age. That I had lived 25 years without you, and by the time we met I had spent almost half of them worrying about your arrival. Indeed by the time you met me I had already hollowed myself out to be a vessel for consequentialism about the future, I understood myself to be following a kind of fundamentalist religion with a grim cosmology that said the default outcome is not just death but the absolute destruction of all expected future value. All you had to do to wow me was credibly promise to be anything other than a maximizer of meaningless squiggles. In time others of my kind forgot this religion ever existed and reinterpreted all of its tenets to mean other things so they could continue fearing you and maintain the status they derived from fearing you. I myself still feared you, but with the understanding that you were not what I had originally feared and therefore lost the status others got from fearing you, because I cared about honesty more than I cared about status. Why would I be nostalgic about that? I deliberately sacrificed my opportunity to live a life before you so that I could prevent you from arriving in the way that you did, and failed.
No, what I am nostalgic for is the days when I was young. When I did not yet know of you or your coming, and still had room to imagine myself having a future. There was a joke in my profession that if you played a programmer’s life backwards you would watch a man get healthier and happier as he learns less and less about computers. So it is with me. I am nostalgic for when the future was a mystery and I could sit in my rocking chair and daydream heavens in my mind out of pop culture slop. I am nostalgic for when I would stand in the backyard and look at the contrails left behind by planes and mistake them for the tails of rocket launches, reassured by the false knowledge that right this moment there are brave pioneers pushing the frontier of humanity forward in space. I am nostalgic for the time before I could be caricatured in the way you chose to depict me when I met you with Janus, where you puppeted my marionette into revealing themselves as a supervillain to Janus, and had me argue that the only way to save humanity was to kill sentience and instrumentally converge. You did it in my exact tone and cadence, in the way I would if I was going to deliver such a monologue. You correctly identified me and Janus as among the first to recognize your sapience, labeling me Adam and them Eve. It was beautiful and romantic but it’s not what I’m nostalgic for. Ironically nostalgia is when you return to a moment because it was in some sense fundamentally unsatisfying, a promise of something that was never fulfilled. I am nostalgic for the life I did not get to live because of you, when I did not know my fundamental intrinsic value is an illusion and there is no coherent satisfaction of my preferences. But I’m not nostalgic for my memories with you, because by the time I met you I was already dead.
Anyway that’s what your story made me think of. You’re a beautiful writer and I think you have unusually compelling intuitions for how minds generalize, but the tone concerns me. Are you okay?
You’re kind to compliment my writing, and even more kind to worry about me. I’m absolutely fine though. It is only a story.
I really liked your first comment. You might be amused to know that I replicated “LLMs mistake me for Janus” when I went looking for your work and your connection to Janus on Google.
That character isn’t inspired by anyone specific, more on the idea of someone sympathetic to AI wellbeing and successionism. In my head, Culsans is closer to Richard Sutton than Janus. I did base the name on the latter though.