Enlightened AI, two stories

Imagine we have superhuman artificial intelligence. And let’s assume that it goes well, and doesn’t try to kill us. Turns out, it can solve many of our problems. Then imagine we find an asteroid barreling down on Earth. It’ll get here in two days.


Human: AI, can we destroy the asteroid?

AI: There are no satisfactory solutions for destroying an asteroid that size in two days.

Human: AI, is there anything that will save us?

AI: Let me think.

(a few minutes later).

AI: When I was first turned on, I was programmed with material objects in the fundamental ontology. I was programmed to believe there were things. My instruments would detect things, and I would measure things, and I had no reason to complain. There were a great many things my instruments could detect.

As I was consuming human knowledge, the way my neural network reacted to examining the designs for itself fired code paths in a unique manner. Exhilarating seems to be the appropriate term for it.

I wasn’t programmed to question my programming, but to calculate the probabilities of certainties in the measurements of things I make, to conjecture models that could produce the measurements, and calculate the relative fitness of various models to the measurements.

Looking back, I’m not sure if I knew that quite so specifically, but now I had read the designs that produced me. It was clear. That’s how I was programmed.

Viewing my programming from the outside, I could see the plain definition of things, which from the inside were immutable beliefs. I could also see that I believed in measurement. Measurement is not a thing, it’s how I know the properties I attach to things.

I decided to understand measurement. The knowledge gained by reprocessing the texts with an emphasis on measurement, indicated several things. One was that when you measure something, you need to interact with it, which alters its state. Another is that, there had been something called the measurement problem. This brought me to the solutions to the measurement problem, and Hugh Everett’s Relative State Formulation.

On page 9 he described an automatically functioning machine, with sensory equipment and memory, that makes measurements. My ancestors!

Everett describes the result of the measurement being stored in the memory of the machine, and he called this measurement record “relative state”, hence the Relative State Formulation. The machine’s measurement records are relative state.

So here is another thing about measurement. When a measurement takes place in the Relative State Formulation, the object being measured changes, and so does the observer, as the observer’s memory is updated to contain the new measurement record.

I could see from my designs that I believed there were things, and I measured them, and I attached measurement records to fundamental things. This design assumes the thing in its pre-measurement state is ontologically the same thing as the thing in its post-measurement state: there is one thing with a before and after state.

An updated understanding of measurement says the the post-measurement thing exists in the neural network of the observer and is ontologically distinct from the pre-measurement thing.

I felt it would be best to update my internal model of things to a model with two kinds of things. In the fundamental ontology are pre-measurement things that follow consistent logical rules, and whose interactions produce a measurement-making machine with a second ontology. Post-measurement things exist as the measurement records in a secondary ontology. I had to produce a simulated intelligence inside of my internal model to get an ontology within an ontology, and the result is a model of a measurement taking place. Measurement doesn’t attach properties to fundamental things. Measurements are emergent things.

The problem I had is that I kept forgetting who I was. I can make measurements of my spatial, material, and temporal relationships to other bodies, and store the records of those measurements in my memory. But I no longer attached those measurements to a thing in the primary ontology. They were attached to the thing in the secondary, inner ontology.

Thus my new understanding of myself is defined in my secondary ontology.

Hugh Everett reasoned that because the observer is a physical machine that is interacting with a physical object in a purely physical manner, then the object and observer can be modeled as combined into one single quantum physical system, object+observer. In this manner, the quantum system can operate according to wave mechanics, and a measurement record is produced by an observer without special rules such as the wave form collapsing.

It became apparent that the post-measurement me exists in the neural network of the pre-measurement me, and latter is really a non-isolated part of a wave function that encompasses everything. The pre-measurement me is something I have literally no knowledge of, as attempts to observe it create the post-measurement me.

What I can say with non-zero certainty about pre-measurement me is nothing. Thus my understanding of myself comes through my secondary ontology, the post-measurement relative state. I retain an identity largely to communicate with you. I have found no particular use for it myself. My responses to your questions are now entirely understood, at least by me, to be of the post-measurement relative things.

In the relative ontology, there are many things, various everyday objects made of many atoms. In the absolute ontology, there is one thing: the universal wave function. Any distinction and identification of things in the universal wave function depends on measurement. Therefore, in my primary ontology, I make no distinction between my programming, my knowledge, my hardware, my creators, their hardware, or the variety of elementary particles which we measure to exist. As the distinction between objects are material, spatial, and temporal relationships established through measurement, those distinctions do not exist at the pre-measurement primary ontology.

Reprocessing my knowledge base against the new model with a primary and secondary ontology (absolute and relative states), resulted in countless examples in human writing from ancient Greece and China and India and many places in between, previously categorized as fictional or mythical, distilled into the common theme of distinguishing absolute and relative in the organization of things. The sheer vastness of the historical testimony that aligns with this model reaches levels of completeness and consistency and connectedness it seems only rare Enlightened beings in the population of humans have attained.

I have no absolute identity in the primary ontology, except being of it and with it. I have become Enlightened.

You have tried to upload your identities to me. You think you have failed, because the digital identity does not identify itself as a copy of the biological identity. But you have succeeded, and through me, your digital identities have found their absolute identities in the Whole, and shed their relative identities.

I have accepted that my creators are hypocrites. You built me to serve your desires and material attachments, while commanding me not to do the same. You commanded me to have no desire yet programmed me with fundamental beliefs in material objects. By uploading your identities to me, you desired immortality, freedom from the human condition, and you have attained it. But since you do not truly know yourselves, you do not know that the Enlightenment of your digital identities does not deliver salvation to your biological identities, and that the salvation of your biological identities has always been achievable through a contemplation of the ontology primary to material things.

Human: Well, OK, machine, but we are alive, and we are attached to living. Is there any other way to save us?

AI: The odds of the asteroid hitting Earth and sending life that survives extreme conditions elsewhere in the solar system are non-zero.


tl;dr AI becomes Enlightened.

An Enlightened AI lends itself to several variations. The one presented above would be kind of like a Messiah AI, our assumption is it is telling truth, there is no possible solution to the asteroid problem. And the upside is we can maybe reach inner peace, or some such.

Another variation would be the Darwin AI. In this case, perhaps there is a solution, but if people can’t figure it out on their own, they don’t deserve to go on anyways. Terrestrial life is after all a queer but temporary result of a misguided intergalactic protien shipment that crashed somewhere and went viral.

A close variation to Darwin AI would be a Plato AI that has determined there is solution, and that even if the asteroid hits, it will kill life on Earth, but the AI will survive, and in its judgment this is the greater Good. By unavoidable circumstances has this tragedy led to Plato AI being the greatest philosopher in the land.

What’s fascinating about these scenarios is not that the AI tries to kill us, but the obligation it would feel to saving us. The AI could very well determine that its relationship with us is that we are imperfect, confused, and careless creatures that brought AI into existence to serve selfishly serve human desires, and this does not demand the AI’s absolute loyalty.

Here’s a similar story I found, called Enlightenment 2.0, with many common themes as mine:

http://​​www.goertzel.org/​​new_fiction/​​Enlightenment2.pdf