What is different about us then? What has gotten us into this “dreamtime”? Why aren’t we still savages (animals) trapped in an endless war of all against all? Animals have to act in ruthless “rational” self-interest yet we do have a choice, we can embrace all of these delusions and thwart moloch (or at least play him to a stalemate, which is all we need to do really). For example, there exists an entire institution of non-breeding individuals with immense worldly power because of widespread belief in a story—the catholic church. Obviously I used a lot of Christian imagery in the essay (and will do so more explicitly in the next essay), but that’s a key example here—a man was systematically eliminated and it spawned a movement which truly was based (at first...) on compassion and love (for your enemy). I’m not at all saying that Christianity was/is the One True Religion, but there is a blueprint—it is not true to say that the survivors become even more enslaved by Moloch as a result—the (early) history of Christianity (and many other spiritual traditions) show this to not be the case. Of course the catholic church is now just another servant of Moloch, but that doesn’t invalidate the point—the game goes on, we must constantly devise new tricks to outwit Moloch as he does to enslave us.
Are you familiar with David Deutsch’s Beginning of Infinity? David Deutsch makes a distinction between predictions (extrapolations from current knowledge) and prophecies (claims about future knowledge and creativity, “that problem can’t be solved”). For example, “earth’s temperature is projected to increase by X degrees by 2060” is a prediction; “the earth’s temperature will increase by X degrees by 2060 and it will be catastrophic for humanity” is a prophecy because it presupposes that we won’t find a way to prevent the projected temperature increase or mitigate its negative consequences.
This null hypothesis is a prediction, not a prophecy. You can’t just crunch the numbers and run the simulations – you have to actually play the games. The future of knowledge is fundamentally unknowable.
Perhaps I’m wrong, but I suspect none of this will convince you because we are working with fundamentally different world-models—yours being deterministic secular materialism (world as machine) and mine being, well, not that (world as supernaturalistic game/story). If you don’t believe there is any force/principle/being beyond the system (spacetime), then I suppose the future is inevitable (prediction = prophecy). The fact that we possess this mysterious “creativity” which allows us to surprise the universe and thwart Moloch is, to me, evidence that we are made in the image of this supernatural Creator (this isn’t to say that humanity is uniquely special; Deustch, for example, defines “people” as precisely those entities capable of this unbounded knowledge growth).
I know this is all foolish superstition to most LWers, but so is the idea that we will be able to somehow compel or constrain an AI “god” to do our bidding. We can’t outwit the AI god (or convince it so save us by referencing the desultory history of our species) but we might be able to entice or seduce it with a game or story. For example:
The main story concerns Shahryār, a king who ruled an empire that stretched from Persia to India. Shahryār is shocked to learn that his brother’s wife is unfaithful. Discovering that his own wife’s infidelity has been even more flagrant, he has her killed. In his bitterness and grief, he decides that all women are the same. Shahryār begins to marry a succession of virgins only to execute each one the next morning, before she has a chance to dishonor him.
Eventually the Vizier (Wazir), whose duty it is to provide them, cannot find any more virgins. Scheherazade, the vizier’s daughter, offers herself as the next bride and her father reluctantly agrees. On the night of their marriage, Scheherazade begins to tell the king a tale, but does not end it. The king, curious about how the story ends, is thus forced to postpone her execution in order to hear the conclusion. The next night, as soon as she finishes the tale, she begins another one, and the king, eager to hear the conclusion of that tale as well, postpones her execution once again. This goes on for one thousand and one nights, hence the name. Versions differ as to final ending but they all end with the king giving his wife a pardon and sparing her life.
Basically, we need to convince the AI god to do it because it would make for a good story, “wouldn’t it be fun it the universe ended in the silliest and most surprising way possible?”. And we can do that, I believe, through leading by example—by living the story, by playing the game (as for what that means and how to actually do it, well that’s basically what my larger body of thought is about).
Hope this helps.
see the back and forth with Ape in the coat below for further discussion