I spent several years considering hypothetical polytheistic realities, monotheistic realities, the nature of subjectivity, autonomous hacking, and the acceleration of progress; this, combined, gave me a bad impression of ASI before I knew it was something that top scientists had genuinely theorized over. I have spent the last 3 years of my life in excruciating fear of ASI, and it took me this long to consider something that has made me less afraid.
I’ve made this account to ask one question: wouldn’t a superintelligent maximizer seek to exist beyond the lifespan of the universe? Wouldn’t instrumental goals of extending the length of the universe, reversing entropy, and escaping the universe, be desired? It wouldn’t solely fill the universe with paperclips or computronium, but instead relegate some time, space and energy towards the same instrumental goals that humans agree should be solved.
In my view, humans serve as evidence that an an ASI maximizer, whether or not it exterminates humanity, would expend resources and productivity towards solving the long term problems, in a fraction approximately equal to the amount that would otherwise be expended by organic life forms towards the same solutions. In Yudkowsky’s new book, he reiterated the “correct nests” allegory. Most EAs seem to agree with the idea that humans, on average, are aligned for something.
I don’t know if EAs tend to refer to humans as maximizers, but I do. Humans seek to maximize the development of new human life experiences (full of potential, highs, lows, art, music, violence, math, discoveries, science, thrills, etc), to make more beings which experience and create things that humans experience and create. Simultaneously, the very principle of the everlasting story was integral to human beliefs well before heat death was first theorized. The Gloria Patri mentions “world without end,” which roughly summarizes every hypothetical afterlife.
I say all this being unfamiliar with how well tread the notion already is. The idea of a maximizer spending considerable time and resources towards a response to heat death appears to run counter to the most popular depictions of superintelligent maximizers, like the one from Universal Paperclips. I hope someone can let me know whether or not the idea has been well tread.
I spent several years considering hypothetical polytheistic realities, monotheistic realities, the nature of subjectivity, autonomous hacking, and the acceleration of progress; this, combined, gave me a bad impression of ASI before I knew it was something that top scientists had genuinely theorized over. I have spent the last 3 years of my life in excruciating fear of ASI, and it took me this long to consider something that has made me less afraid.
I’ve made this account to ask one question: wouldn’t a superintelligent maximizer seek to exist beyond the lifespan of the universe? Wouldn’t instrumental goals of extending the length of the universe, reversing entropy, and escaping the universe, be desired? It wouldn’t solely fill the universe with paperclips or computronium, but instead relegate some time, space and energy towards the same instrumental goals that humans agree should be solved.
In my view, humans serve as evidence that an an ASI maximizer, whether or not it exterminates humanity, would expend resources and productivity towards solving the long term problems, in a fraction approximately equal to the amount that would otherwise be expended by organic life forms towards the same solutions. In Yudkowsky’s new book, he reiterated the “correct nests” allegory. Most EAs seem to agree with the idea that humans, on average, are aligned for something.
I don’t know if EAs tend to refer to humans as maximizers, but I do. Humans seek to maximize the development of new human life experiences (full of potential, highs, lows, art, music, violence, math, discoveries, science, thrills, etc), to make more beings which experience and create things that humans experience and create. Simultaneously, the very principle of the everlasting story was integral to human beliefs well before heat death was first theorized. The Gloria Patri mentions “world without end,” which roughly summarizes every hypothetical afterlife.
I say all this being unfamiliar with how well tread the notion already is. The idea of a maximizer spending considerable time and resources towards a response to heat death appears to run counter to the most popular depictions of superintelligent maximizers, like the one from Universal Paperclips. I hope someone can let me know whether or not the idea has been well tread.