I am a psychotherapist helping people working on AI safety.
Check out my website to learn more about me or book a session.
If you would like to discuss with me other ways to get professional help, you can also book a short call on my website.
I also make community events for senior AI governance specialists.
The reactor meltdown on a Soviet submarine was not posing an existential threat. In the worst case, it would be a little version of Chernobyl. We might compare it to an AI which causes some serious problems, like a stock market crash, but not existential ones. And the movie is not a threat at all.
”The question is how plausible it is to generate situations that are scary enough to be useful, but under enough control to be safe.”
That is a great summary of what I wanted to say!
I agree that fearmongering is thin ice, and can easily backfire, and it must be done carefully and ethically, but is it worse than the alternative in which people are unaware of AGI-related risks? I don’t think that anybody can say with certainty
ChatGPT was recently launched, and it is so powerful, that it made me think that the problem of a misuse of a powerful AI It’s a very powerful tool. No one really knows how to use it, but I am sure, we will soon see it used as a tool for unpleasant things
But I also see more and more of perception of AI as a live entity with agency. People are having conversations with ChatGPT as with a human
First, your article is very insightful and well-structured, and totally like it.
But there is one thing that bugs me.
I am a person new to AI alignment field, and recently, I realized (maybe by mistake) that there is very hard to find a long-term financially stable full-time job in AI field-building.
For me, it basically means that only a tiny amount of people consider AI alignment important enough to pay money to decrease P(doom). And at the same time, here we are talking about possibility of doom within next 10 or 20 years. For me it is all a bit crazy
I also think that sooner or later, when AIs will become more and more capable, and, either some large Chernobyl-like tragedy caused by AI will happen, or some AI will become so powerful that it will horrify people. In my opinion, probability of that is very high. I already see how ChatGPT spread some fear. And fear might spread like a wildfire. If it will happen too late for governments to react thoughtfully, it will introduce a large amount of risk and uncertainty. In my opinion, too much risk and uncertainty.
So, in my opinion, even if we will educate the public and promote government regulation, and if AGI will appear before 2030, then government policies might suck. But if we will not do it, they might suck much more and it is even more dangerous.
Good post
I have similar thoughts. I believe that at one moment, fears about TAI will spread like a wildfire, and the field will get a giant stream of people, money and policies, and it is hard to feel from today
Good post, but there is a big disbalance in human-ants relationships.
If people could communicate with ants, nothing would stop humans to make ants suffer if it made the deal better for humans because of a power disbalance.
For example, domesticated chickens live in very crowded and stinky conditions, and their average lifespan is a month after which they are killed. Not a particularly good living conditions.
People just care about profitability do it just because they can.
I 100% agree with you.
I am a person entering the field right now, I also know several people in a position similar to mine, and there are just no positions for people like me, even though I think I am very proactive and have valuable experience
It does, and it causes a lot of problems, so I would prefer to avoid such problems with AIs
Also, I believe that an advanced AI will be much more capable in terms of deception and manipulation than an average human
Thank you for your comment and everything you mentioned in it. I am a psychologist entering the field of AI policy-making, and I am starving for content like this
I agree. We have problems with emotional attachment to humans all the time, but humans are more or less predictable, not too powerful, and usually not so great at manipulations
One thing helping me to preserve hope is the fact that there are so many unknown variables about AGI and how humanity will respond to it, that I don’t think that any current-day prediction is worth a lot.
Although I must admit, doomers like Connor Leahy and Eliezer Yudkovsky might be extremely persuasive but they also don’t know many important things about the future and they are also full of cognitive biases. All of this makes me tell myself a mantra “There is still hope that we might win”.
I am not sure whether this is the best way to think about these risks but I feel like if I’ll give it up, it is a straightforward path to existential anxiety and misery, so I try not to question it too much.
Thanks for your view on doomerism and your thoughts on the framing of a hopre
What are examples of “knowledge of building systems that are broadly beneficial and safe while operating in the human capabilities regime?”
I assume the mentioned systems are institutions like courts, government, corporations, or universities
Charlotte thinks that humans and advanced AIs are universal Turing machines, so predicting capabilities is not about whether a capability is present at all, but whether it is feasible in finite time with a low enough error rate.
I have a similar thought. If AI has human-level capabilities, and a part of its job is to write texts, but it writes large texts in seconds and can do it 24⁄7, then is it still within the range of human capabilities?
I absolutely agree with the conclusion. Everything is moving so fast.
I hope, these advances will cause massive interest in the alignment problem from all sorts of actors, and even if OpenAI are talking about safety (and recently they started talking about it quite often) in a large part because of PR reasons, it still means that they think, society is concerned about the progress which is a good sign.
I think, we are entering a black swan and it’s hard to predict anything.
I agree
In my opinion, this methodology will be a great way for a model to learn how to persuade humans and exploit their biases because this way model might learn these biases not just from the data it collected but also fine-tune its understanding by testing its own hypotheses