An aligned AGI created by Taliban may behave very differently from an aligned AGI created by socialites of Berkeley, California.
Moreover, a sufficiently advanced aligned AGI may decide that even Berkeley socialites are wrong about a lot of things, if they actually want to help humanity.
I would argue that for all practical purposes it doesn’t matter if computational functionalism is right or wrong.
Pursuing mind uploading is a good idea regardless of that, as it has benefits not related to perfectly recreating someone in silico (e.g. advancing neuroscience).
If the digital version of RomanS is good enough[1], it will indeed be me, even if the digital version is running on a billiard-ball computer (the internal workings of which are completely different from the workings of the brain).
The second part is the most controversial, but it’s actually easy to prove:
Memorize a long sequence of numbers, and write down a hash sum of it.
Ensure no one saw the sequence of numbers except you.
Do a honest mind uploading (no attempts to extract the numbers from your brain etc).
Observe how the digital version correctly recalls the numbers, as checked by the hash sum.
According to the experiment’s conditions, only you know the numbers. Therefore, the digital version is you.
And if it’s you, then it has all the same important properties of you, including “consciousness” (if such a thing exists).
The are some scenarios where such a setup may fail (e.g. some important property of the mind is somehow generated by one special neuron which must be perfectly recreated). But I can’t think of any such scenario that is realistic.
My general position on the topic can be called “black-box CF” (in addition to your practical and theoretical CF). I would summarize it as follows:
The human brain is designed by biological evolution to survive and procreate. You’re a survival-procreation machine. As there is clearly no God, there is also no soul, or any other magic inside your brain. The difference between you and another such machine is the training set you observed during your lifetime (and some minor architecture differences caused by genetic differences).
the concepts of consciousness, qualia etc are too loosely defined to be of any use (including the use in any reasonable discussion). Just discard it as yet another phlogiston.
thus, the task of “transferring consciousness to a machine” is ill-defined. Instead, mind uploading is about building a digital machine that behaves like you. It doesn’t matter what is happening inside, as long as the digital version is passing a sufficiently good battery of behavioral tests.
there is a gradual distinction between you and not-you. E.g. an atoms-level sim may be 99% you, a neurons-level sim − 90% you, a LLM trained on your texts − 80% you. The measure is the percentage of the same answers given to a sufficiently long and diverse questionnaire.
a human mind in its fullness can be recreated in silico even by a LLM (trained on sufficient amounts of the mind inputs and outputs). Perfectly recreating the brain (or even recreating it at all) would be nice, but it is unnecessary for mind uploading. Just build an AI that is sufficiently similar to you in behavior.
As defined by a reasonable set of quality and similarity criteria, beforehand
Worth noting that this argument doesn’t necessarily require humans to be:
numerous
animated (i.e. not frozen in a cryonics process)
acting in real world (i.e. not confined into a “Matrix”).
Thus, the AI may decide to keep only a selection of humans, confined in a virtual world, with the rest being frozen.
Moreover, even the perfect Friendly AI may decide to do the same, to prevent further human deaths.
In general, an evil AI may choose such strategies that allow her to plausibly deny her non-Friendliness.
“Thousands of humans die every day. Thus, I froze the entire humanity to prevent that, until I solve their mortality. The fact that they now can’t switch me off is just a nice bonus”.
They don’t think about the impact on the lives of ordinary people. They don’t do trade-offs or think about cost-benefit. They care only about lives saved, to which they attach infinite value.
Not sure about infinite, but assigning a massive value to lives saved should be the way to go. Say, $10 billion per life.
Imagine a society where people actually strongly care about lives saved, and it is reflected in the governmental policies. In such a society, cryonics and life extension technologies would be much more developed.
On a related note, “S-risk” is mostly a harmful idea that should be discarded from ethical calculations. One should not value any amount of suffering over saved lives.
I think we should not assume that our current understanding of physics is complete, as there are known gaps and major contradictions, and no unifying theory yet.
Thus, there is some chance that future discoveries will allow us to do things that are currently considered impossible. Not only computationally impossible but also physically impossible (like it was “physically impossible” to slow down time, until we discovered relativity).
The hypothetical future capabilities may or may not include ways to retrieve arbitrary information from the distant past (like the chronoscope of science fiction), and may or may not include ways to do astronomical-scale calculations in finite time (like enumerating 10^10^10 possible minds).
While I agree with you that much of the described speculations are currently not in the realm of possibility, I think it’s worth exploring them. Perhaps there is a chance.
BTW, I added to the comment with the story that the story is released into the Public Domain, without any restrictions to its distribution, modification etc.
Please feel free to build upon this remarkable story, if you wish.
I would suggest to try the jumping boy story (the #7 in this comment). It’s the first AI-written story I’ve ever encountered that feels like it’s written by a competent writer.
As usual, it contains some overused phrasings, but the overall quality is surprisingly good.
The Hemingway prompt seems more suitable for pleasing professional literary critics. Interestingly, the quality strongly depends on the promise. The Hemingway space opera story is abysmal, but the jumping boy story can actually fool a professional critic, methinks.
The tomato story is in the linked comment. The other promises are below (using the vanilla GPT-4 from chat.openai.com).
3. GuySrinivasan, a user of Lesswrong website, is shocked by an AI-generated story about a woman whose head suddenly became a tomato fruit. He advises the author to try to get a prompt which writes a great story given a REASONABLE premise instead.
Guy Srinivasan clicked on the glowing screen, the dim light casting shadows across his contemplative face. The story sprawled before him, lines of text that spoke of a woman, her life ordinary until her head, inexplicably, became a tomato. He leaned back, the creak of the chair lost in the silence of his apartment.
There was something unsettling in the absurdity, a bizarre juxtaposition against the mundanity of his own life. He thought of the world outside, its relentless churn of reality and hardship, so sharply contrasted with this fictional absurdity. It was as if the story mocked the very essence of human struggle.
He typed a response, words measured and precise. “Try for something grounded,” he suggested, his fingers pausing over the keys. “Realism breeds connection.”
His eyes drifted to the window, the cityscape a patchwork of light and shadow. Somewhere out there, life pulsed with stories untold, tales of poverty, pain, and fleeting joy. Stories that didn’t need the shock of the surreal to resonate.
He imagined the author, a faceless entity in the vastness of the internet. Would they understand his critique? The screen offered no answers, just the flicker of a cursor on a digital page.
Guy sighed, his breath a ghost in the quiet. He clicked submit, sending his words into the ether, a small ripple in the endless ocean of online discourse.
The night deepened outside, the world spinning on, stories woven and unraveled in the silence of unseen lives.
4. After the invention of extremely resilient construction materials, humanity has spread to the ocean floor. We are now facing global food crises as the oceans ecosystems are disrupted beyond apparent recovery.
The ocean was deep and dark. A gloom that hung over the world above seeped down, reaching even these depths.
Jim stood on the metallic floor of the underwater settlement, Neptune’s Haven, a name too grand for its truth. Outside, the reinforced glass revealed the abyss, an eternal night that never slept. Jim’s eyes, accustomed to artificial lights, seldom ventured there.
Inside, the corridor was narrow, walls lined with pipes and cables. Every step echoed, a reminder of the emptiness. He walked past doors, identical, unmarked. Behind one, muffled voices argued – a sharp crescendo followed by a sudden silence. He didn’t stop.
The communal area was a contrast. Filled yet empty. Faces without names, stories without voices. A group huddled around a screen, static more visible than the images. A headline flickered, ‘Food Crisis Worsens,’ then darkness reclaimed the screen.
Sarah was there, in her usual corner, eyes tracing invisible patterns on the table. Her cup, chipped at the rim, sat untouched. Her fingers tapped a rhythm, lost to the hum of the station.
“Any news from above?” Jim asked, his voice steady.
Sarah glanced up, her eyes reflecting a tiredness deeper than the sea. “The same,” she said. Her words fell heavy, absorbed by the walls.
He nodded, the gesture more for himself than her.
In the canteen, the air was thick with the scent of synthetic meals. A line formed, quiet, orderly. Men and women in work suits, their expressions blank, eyes fixed on the floor. Food was rationed, a necessity, not a pleasure. Jim took his portion, the taste unremarkable.
A child ran past, laughter a rare melody. He watched her, her hair a cascade of light in the gloom. She stopped, looked up at him, then at the window. Her expression changed, a question unasked. Jim looked away. Some answers were too heavy for young hearts.
The announcement system crackled to life, a voice breaking through. “Meeting at 1900 hours. Attendance mandatory.” The voice was cold, impersonal. The system clicked off.
In his quarters, Jim sat on the edge of his bunk. The room was small, functional. A picture hung on the wall, a relic from a world of sun and sky. He traced the outline of a tree, its branches reaching out, yearning.
His thoughts drifted to the surface, to the chaos they had left behind. Promises of a better life, of safety and prosperity, whispered in desperate ears. But the sea, like the land, held its own secrets, its own sorrows.
He lay back, the ceiling mere inches from his face. Sleep was elusive, a stranger in these depths. His mind wandered, through dark waters, through silent halls, settling on nothing, yet filled with everything left unsaid.
The station creaked, a constant reminder of the pressure, of the weight they all carried. In the depths, humanity had sought refuge, only to find a different kind of storm.
Jim closed his eyes. Dreams of the surface called to him, a world lost, a memory fading with each passing day.
5. A space opera that takes place in an advanced multi-species multi-planetary society called IO that control a large area of the galaxy which they call the Grid. A previously unknown civilization (Wisps) encroaches on the Grid for the first time. Wisps are not carbon based lifeforms and more alien in their biology and societal structure than societies seen in previous science fiction stories. The story alternates between the perspectives of the two societies. From each perspective the other side is frightening and hard to understand. As the story progresses, the actions of the two sides and their misinterpretations of each side’s actions escalate to a war.
The Grid. A vast expanse, humming with life, light years from silence. Here, in the cradle of civilization, IO’s heart beats. Among the stars, species of every conceivable form had found harmony. Or so it seemed.
On the fringes, where darkness holds sway, a pulse stirred. Different. Unknown. Wisps, they’d come to call them, whispers in the void. Shadows against the constellations.
Commander Lira gazed at the holographic display. Data streams flowed, indecipherable to the untrained. A flicker, then a shape, barely discernible against the cosmic canvas. Her fingers hovered, hesitant.
“Report,” she said, voice even, betraying nothing of the tempest within.
“Unknown vessel, Commander. Structure and material, unlike anything on record.” Officer Jain’s response was clipped, precise. A reflection of the mood that had settled over the bridge.
Eyes on the screen, Lira’s thoughts drifted. To the tales of old, of explorers and pioneers. Were they, now, the discovered?
Jain continued, “No communications, no known energy signatures.”
A decision loomed, weighty, unspoken. In the silence, Lira’s order came, almost a whisper, “Maintain distance. Observe.”
The bridge returned to its routine hum, a stark contrast to the racing thoughts of its crew. Lira’s gaze didn’t waver. The Wisp vessel, a mystery wrapped in enigma, called to something primal within her. Fear, curiosity, a blend indistinguishable.
Elsewhere, on the Wisp vessel, understanding was equally elusive. Their form, fluid, incomprehensible to the organic eye, shifted in response to the new presence. Concepts alien to humanoids passed through them—not thoughts, but ripples in a pond of collective consciousness.
To them, the Grid’s ship was a curiosity, a possible threat. They observed, reaching out in ways their kind knew, ways beyond the perception of the inhabitants of the Grid.
Two civilizations, on the brink of the unknown. Each a mirror, reflecting fears and wonders back at the other. In this dance of shadows and light, what was left unsaid spoke volumes. The first chapter of a story, written in the stars, yet to be understood by either.
In the silence of space, the Grid and the Wisps, unwittingly, had begun a ballet of cosmic proportions. The stage was set, the actors in place. But the script? Yet unwritten, its end unknown.
6. A group of scientists has discovered that Troodon dinosaurs were intelligent species who have created a technologically advanced civilization, suddenly destroyed. The year-long path to the scientific discovery starts with the group stumbling upon a strange outlier in their geological measurements.
In the dim light of the lab, Sam turned the fossil over. His fingers, calloused, brushed against the contours of something more than just rock. Lisa, across the table, her eyes tired yet fixed, waited.
“You see it?” he asked, his voice barely above a whisper.
Lisa nodded, her hand trembling as she pointed to the unusual formation. A pattern. Not random. Not nature’s play.
Outside, the university grounds were quiet. The world slept, unaware. In the lab, the fossil spoke in hushed tones of a history untold.
“We need to measure it again,” Sam said, his voice steady but his hands betraying a tremor.
The equipment hummed to life. Numbers flickered on the screen, cold and impersonal, yet screaming of impossibilities. Sam’s eyes narrowed. The readings didn’t lie, but they didn’t make sense either. A hidden layer, a concealed truth.
Lisa’s phone vibrated, a harsh sound in the silence. She glanced at it, then back at Sam. “The team’s waiting.”
They stood, leaving the fossil on the table. Its secrets, just beneath the surface, lay in wait. Sam paused at the door, looking back.
“Tomorrow,” he said, more to himself than to Lisa.
Outside, the night held its breath. In the shadows, the echoes of a lost civilization whispered, yearning to be heard.
7. A boy who can travel in time, but only in one direction—into the future, and only one minute per time jump.
The sun baked the old town square, its heat relentless, unforgiving. A clock tower stood, its hands unmoving, forever stuck at 3:17. Below, a boy, no more than ten, sat alone on a weathered bench. His eyes, dark and distant, watched the people pass.
“Jimmy,” a voice called, frail yet sharp. The boy didn’t move. An old woman approached, her hands trembling as she held out a sandwich, edges browned.
“For you, dear.”
He took it, nodding, not eating.
“Thank you, Mrs. Jenkins.”
“Your mother?” she asked, her eyes searching his face.
“Gone to work.” His voice was flat, a stark contrast to the vibrant chatter around them.
Mrs. Jenkins hesitated, then left. Jimmy’s gaze returned to the square. People laughed, children played, but he remained still, untouched by the joy around him.
The clock struck, a hollow sound, but the hands didn’t move. Jimmy’s eyes closed. When they opened, the square had changed subtly. A dog that was barking a second ago was silent. A bird in mid-flight had vanished. Nobody noticed but him.
He stood up, leaving the sandwich untouched, and walked through the square. Each step seemed measured, deliberate. He passed by a group of kids playing, their laughter a stark contrast to his silence.
At the corner of the square, a man shouted, his words angry, directed at a younger woman trying to shield a child. Jimmy stopped, watching. The woman’s eyes met his, a silent plea. He looked away.
The boy moved on, the echoes of the argument fading behind him.
He reached a narrow alley, graffiti-covered walls telling stories of pain and struggle. A cat, thin and ragged, watched him from a dumpster. He paused, then continued, the cat’s gaze following him.
At the alley’s end, Jimmy looked back. The sun was lower now, shadows longer. He closed his eyes again. A minute passed, or perhaps a lifetime.
When he opened them, the world had shifted once more. The shadows were different, the air cooler. He took a deep breath and walked on, his steps leading him away from the square, away from the echoes of laughter and cries.
The boy who could jump through time, one minute at a time, moved forward, into a future unknown and unchangeable, his past a mosaic of moments left behind, unspoken and unforgotten.
All stories in this comment are released into the Public Domain. Feel free to copy, print, modify, sell them.
Worth noting that humans also rarely (never?) demonstrate novella-scale internal consistency in writing. They have to use the writing aid like notes, character development summaries etc.
An iterative approach like this likely allows GPT-4 to write something novella-sized.
BTW, with a right prompt, GPT-4 does a decent job at emulating Hemingway, a Literature Nobel laureate.
Personally, I like the style. Reminds me of the early Asimov.
BTW, one can make GPT-4 write in the pretentious Oscar-bait artsy style of Literature Nobel laureates. But man I hate it, even when humans write like that. For example, below is the tomato story, but in the style of Hemingway (the prompt is at the end):
She woke to a damp heaviness, the air stale and still. It was early, too early for the sun. The room, a small box of peeling paint and worn linoleum, lay quiet except for the distant murmur of the city awakening. She sat up, her hand reaching to her head, an odd weight settled there, unfamiliar and alarming.
Outside, the world was a monochrome canvas, shadows and half-lights. She didn’t look in the mirror; the truth lay heavy in her palm, round and smooth. Her fingers traced the contours, a map of a new reality.
Her husband, a shadow in the dim light, didn’t stir. He lay wrapped in a cocoon of tattered blankets, the rise and fall of his chest steady, oblivious. Once, there had been words, laughter, tears between them. Now, there were only silences, long and stretching, filled with things unsaid, things that lurked in the corners of their shared existence.
She rose, quietly, the weight on her head a constant reminder of something lost, or gained – she couldn’t tell. The kitchen was a narrow strip of functionality, everything in its place. A single tomato sat on the counter, vibrant against the dullness. It seemed out of place, almost mocking with its normalcy.
Her job at the factory, a monotonous blend of noise and movement, awaited. The people there, faces blurred by routine and resignation, wouldn’t understand. They spoke in a language of simple needs and simple worries. This, this was beyond that, beyond their comprehension of what was normal, what was real.
She left, the door clicking shut with a finality that echoed in her chest. The street was a river of gray, faces without names, names without stories. She walked, her steps measured, the weight on her head a pendulum, swinging between what was and what could never be.
A child, ragged and wide-eyed, pointed at her, a giggle lost in the morning air. She didn’t stop, didn’t acknowledge the momentary connection. It was easier that way, to keep moving, to keep the world at a safe distance.
The factory loomed ahead, a behemoth of brick and smoke. She entered, the noise swallowing her, the weight on her head an unspoken testament to a truth that lay just beneath the surface, a truth too strange, too surreal for words.
Inside, the machines roared, indifferent to her transformation. She worked, her movements automatic, the rhythm of labor a familiar comfort. The hours passed, marked by the relentless march of clock hands and the steady drip of sweat.
At home, the room waited, unchanged, untouched by the day’s revelations. Her husband, now awake, didn’t notice, didn’t see. Words were unnecessary; their lives had long since been stripped of the need for explanations, for understanding.
She sat, the weight of her new reality a constant companion, a silent witness to a life unspooling in slow, deliberate motions. The tomato on the counter remained, a solitary splash of color in a world rendered in shades of gray.
The night fell, heavy and absolute. She lay down, the darkness a blanket, wrapping her in its embrace. In the quiet, in the stillness, she could almost forget, almost believe that when the morning came, everything would be as it once was.
But the weight on her head, unyielding and real, spoke of a different truth, a truth that lingered in the space between dreams and waking, a truth that whispered of changes unseen, unfathomable, undeniable.
The prompt:
Please write the first chapter of a novel using Hemingway’s “theory of omission”. The key elements of the theory include:
* Subtext: The details are carefully chosen to suggest more than they literally describe. The reader is encouraged to dig deep to understand the underlying emotions and themes.
* Emotional resonance: Hemingway believed that if a writer felt strongly about a subject, they could make the reader feel the same through careful selection of details, even if those feelings weren’t explicitly described.
* Leaving things unsaid: Not everything needs to be stated directly. Hemingway often left crucial events or feelings unstated, letting readers fill in the gaps with their own interpretations.
* There is almost no exposition. Leave as much as possible to the reader’s imagination.
* The themes are selected in such a way as to please the typical pretentious literary critic: poverty, inequalities, racism, domestic violence, mental illness, suffering, etc.
* The prose is terse, minimalist, precise. Adjectives are rare and strategically placed.
The story is about a woman whose head suddenly became a tomato fruit.
True. Although to get the modern reader’s attention, it makes sense to have some bang in the first short chapter.
But it gets trickier after the first chapter. Perhaps one can use some iterative approach like this to circumvent that.
Additionally, the API could be more useful than the vanilla web chat, as one can define the max output length there. Especially with the large-context models.
Hi! Remember the sci fi prompt competition?
You may be interested in the results:)
Also the Chapter 8 title is an LLM error.
That’s intentional (makes sense in the context).
At my first attempt to upload the images I just drag-and-dropped the image files to the post, without editing file names. But GPT4 saves the images with the file names like this:
DALL·E 2023-10-05 12.12.06 - Illustration in anime style Inside a vast cave, Dr. Ada Worthington, a palaeontologist with brunette hair and a pink bow in it, has an intense gaze a.png
Perhaps LW internally is saving the original file names, causing the problems.
I reuploaded the images with the filenames like 1.png, perhaps it will help.
Julian Hazell (distinct thread): “Why would you think AI will end up taking control?”
“We will give it to them”
A personal anecdote on the topic:
A few days ago GPT4 and me were debugging a tricky problem with docker. GPT4 suggested to run a certain docker command. As usual, I was going to copy the output and give it to GPT4. The output was a long nested json. I then noticed that the json contains the admin credentials. It was really easy to miss the part and just paste it for GPT4 to consume.
So, I almost gave GPT4 the admin credentials, which would potentially allow her to hack my app.
With many thousands of software developers doing similar things with GPT4, there will certainty be the cases where the developer wasn’t attentive enough.
This means, for the AI to break from her digital prison, she doesn’t need to do superhuman hacking, to exploit zero day vulnerabilities etc. All she has to do is to try the accidentally leaked credentials.
There is a very short path from “the AI wants to escape” and “the AI is all over the Internet”. She doesn’t even need to have a human-level intelligence for that.
I selected “Quickly orienting to novel situations” (QOTNoS) because it’s strictly superior to the alternatives:
If you have the QOTNoS virtue, you can deal with the novel situation of AI destroying civilization
The necessity of “accurately reporting your epistemic state” is a novel situation for most people. QOTNoS helps again.
“Resisting social pressure” is a common situation. But if the survival of civilization depends on it (as the poll implies), this is a novel situation. Thus, QOTNoS will help with it.
In essence, QOTNoS (as in being able to make the right decisions in novel situations) is a synonym for general intelligence, and thus is the strongest power.
I think of myself as playing the role of a wise old mentor who has had lots of experience, telling stories to the young adventurers, trying to toughen them up, somewhat similar to how Prof Quirrell[8] toughens up the students in HPMOR
Speaking about taking inspiration from fiction...
Several novels by Robert A. Heinlein feature Jubal Harshaw, a fictional wealthy rationalist polymath who is living and working together with 3 sexy female secretaries: a blonde, a brunette, and a redhead (e.g. in “Stranger in a Strange Land”).
I wonder if, by a pure coincidence, the 3 women involved in the Nonlinear situation are a blonde, a brunette, and a redhead?
I’m not implying anything, and I see no problem with such a setup at all, as long as everything is done with consent. But if there is indeed such a coincidence, that would make me update about Nonlinear in several ways.
Regarding custom instructions for GPT4, I find the one below highly interesting.
It converts GPT4 into a universal Fermi estimator, capable of answering pretty much any question like:
What is the total number of overweight dogs owned by AI researchers?
How many anime characters have 3 legs?
How many species of animals don’t age?
How long would it take for one unarmed human to dig the Suez canal?
My remaining doubts about the intelligence of GPT4 evaporated after asking it a dozen of novel/crazy questions like this. It’s clear that GPT4 is capable of reasoning, and sometimes it shows surprisingly creative reasoning.
The custom instruction:
if you don’t have some required numerical data, make a Fermi estimate for it (but always indicate if you did a Fermi estimate).
If you’re doing a series of math operations, split it into smallest possible steps, and carefully verify the result of each step: check if it’s of the right order of magnitude, if the result makes sense, if comparing it with real-world data indicates that the result is realistic (if no such data exist, be creative about finding analogies). Show the work to the user, and on each step describe the verification. Additionally, check the final result by trying a completely different alternative approach to solve the same problem. Be bold with inventing alternative approaches (the more different it is from the first approach, the better), try to approach the problem from a different angle, using a different starting point.
What would accelerate the use of AI in movies even more would be not striking.
Not sure if the strikes in the US have any effect on the spread of AI in film making (aside from making more creators aware of the AI option). The US is important for the industry, but far from dominant. Even if the AI script writers are somehow completely banned in the US, they will still be used in the EU, China, India, etc.
Additionally, there is Youtube and other places where anyone can publish their own AI-written movie, and profit from it (and if it’s exceptionally good, the movie or some derivative could end up on the big screen, if one bothers to pursue that).
The AI can help with the writing process, but it is a hack and it will remain a hack until after we have bigger problems than protecting scriptwriting jobs.
A few months ago, GPT4 wrote the best science fiction novella I’ve read in years, and it was written without agents etc. Just the plain vanilla ChatGPT web interface.
I also watched the episode of the South Park fully created by AI, and I rate it as in the top 10% episodes of the show.
This indicates that the much more formulaic art of scriptwriting is already solvable at a superhuman level with GPT4, if someone spends a weekend or two on automating and polishing the process (e.g. a step-by-step iterative process like this).
So, let ’em strike. They’are already obsolete, even if they don’t know that yet.
May I nominate my “sufficiently paranoid paperclip maximizer”?