Greg C

Karma: 245

Greg C Mar 30, 2023, 7:32 AM
1 point
0
in reply to: Noosphere89’s comment on: FLI open letter: Pause giant AI experiments
Yeah, they work well enough at this (~human) level. But no current alignment techniques are scalable to superhuman AI. I’m worried that basically all of the doom flows through an asymptote of imperfect alignment. I can’t see how this doesn’t happen, short of some “miracle”.

Greg C Mar 30, 2023, 7:25 AM
3 points
2
in reply to: Gerald Monroe’s comment on: FLI open letter: Pause giant AI experiments
the tiniest advantage compounds until one party has an overwhelming lead.
This, but x1000 to what you are thinking. I don’t think we have any realistic chance of approximate parity between the first and second movers. The speed that the first mover will be thinking makes this so. Say GPT-6 is smarter at everything, even by a little bit, compared to everything else on the planet (humans, other AIs). It’s copied itself 1000 times, and each copy is thinking 10,000,000 times faster than a human. We will essentially be like rocks to it, operating on geological time periods. It can work out how to disassemble our environment (including an unfathomable number of contingencies against counter strike) over subjective decades or centuries of human-equivalent thinking time before your sentinal AI protectors even pick up it’s activity.

Greg C Mar 29, 2023, 11:18 PM
1 point
0
in reply to: Portia’s comment on: FLI open letter: Pause giant AI experiments
I also think that compared with other AIs, LLMs may have more potential for being raised friendly and collaborative, as we can interact with them the way we do with humans, reusing known recipes. Compared with other forms of extremely large neural nets and machine learning, they are more transparent and accessible. Of all the routes to AGI we could take, I think this might be one of the better ones.
This is an illusion. We are prone to anthropomorphise chatbots. Under the hood they are completely alien. Lovecraftian monsters, only made of tons of inscrutable linear algebra. We are facing a digital alien invasion, that will ultimately move at speeds we can’t begin to keep up with.

Greg C Mar 29, 2023, 10:56 PM
1 point
0
in reply to: Gerald Monroe’s comment on: FLI open letter: Pause giant AI experiments
Ultimately, it doesn’t matter which monkey gets the poison banana. We’re all dead either way. This is much worse than nukes, in that we really can’t risk even one (intelligence) explosion.

Greg C Mar 29, 2023, 10:36 PM
1 point
0
in reply to: Portia’s comment on: FLI open letter: Pause giant AI experiments
We can but hope they will see sense (as will the US government—and it’s worth considering that in hindsight, maybe they were actually the baddies when it came to nuclear escalation). There is an iceberg on the horizon. It’s not the time to be fighting over revenue from deckchair rentals, or who gets to specify their arrangement. There’s geopolitical recklessness, and there’s suicide. Putin and Xi aren’t suicidal.

Greg C Mar 29, 2023, 8:32 PM
1 point
0
in reply to: Gerald Monroe’s comment on: FLI open letter: Pause giant AI experiments
Look, I agree re “negative of entropy, aging, dictators killing us eventually”, and a chance of positive outcome, but right now I think the balance is approximately like the above payoff matrix over the next 5-10 years, without a global moratorium (i.e. the positive outcome is very unlikely unless we take a decade or two to pause and think/work on alignment). I’d love to live in something akin to Iain M Banks’ culture, but we need to get through this acute risk period first, to stand any chance of that.

Do you think Drexler’s CAIS is straightforwardly controllable? Why? What’s to stop it being amalgamated into more powerful, less controllable systems? “People” don’t need to make them globally agentic. That can happen automatically via Basic AI Drives and Mesaoptimisation once thresholds in optimisation power are reached.

I’m worried that actually, Alignment might well turn out to be impossible. Maybe a moratorium will allow for such impossibility proofs to be established. What then?

Greg C Mar 29, 2023, 7:06 PM
9 points
10
in reply to: Gerald Monroe’s comment on: FLI open letter: Pause giant AI experiments
Except the risk of igniting the atmosphere with the Trinity test is judged to be ~10%. It’s not “you slow down, and let us win”, it’s “we all slow down, or we all die”. This is not a Prisoners Dilema:

[Image credit]

Greg C Mar 29, 2023, 10:57 AM
5 points
0
in reply to: Rafael Harth’s comment on: FLI open letter: Pause giant AI experiments
From the GPT-4 announcement: “We’ve also been using GPT-4 internally, with great impact on functions like support, sales, content moderation, and programming.” (and I’m making the reasonable assumption that they will naturally be working on GPT-5 after GPT-4).

Greg C Mar 29, 2023, 10:30 AM
6 points
5
in reply to: konstantin’s comment on: FLI open letter: Pause giant AI experiments
I think we are already too close for comfort to x-risky systems. GPT-4 is being used to speed up development of GPT-5 already. If GPT-5 can make GPT-6, that’s game over. How confident are you that this couldn’t happen?

Greg C Mar 29, 2023, 10:26 AM
13 points
7
in reply to: konstantin’s comment on: FLI open letter: Pause giant AI experiments
GPT-4 was rushed, and the OpenAI Plugin store. Things are moving far too fast for comfort. I think we can forgive this response for being rushed. It’s good to have some significant opposition working on the brakes to the runaway existential catastrophe train that we’ve all been put on.

Greg C Mar 29, 2023, 10:18 AM
1 point
0
in reply to: Gerald Monroe’s comment on: FLI open letter: Pause giant AI experiments
Why do you think it only applies to the US? It applies to the whole world. It says “all AI labs”, and “govenrments”. I hope the top signatories are reaching out to labs in China and other countries. And the UN for that matter. There’s no reason why they wouldn’t also agree. We need a global moratorium on AGI.

Greg C Sep 5, 2022, 9:33 AM
2 points
on: A Quick List of Some Problems in AI Alignment As A Field
Here’s a (failure?) mode that I and others are already in, but might be too embarrassed to write about: taking weird career/financial risks, in order to obtain the financial security, to work on alignment full-time...

I’d be more glad if I saw non-academic noob-friendly programs that pay people, with little legible evidence of their abilities, to upskill full-time.
CEEALAR offers this (free accommodation and food, and a moderate stipend), and was set up to avoid the failure mode mentioned (not just for alignment, for EA in general).

Greg C Aug 25, 2022, 9:13 AM
3 points
on: A descriptive, not prescriptive, overview of current AI Alignment Research
This is very cool! For archiving and rebuilding after a global catastrophe, how easy would this be to port to Kiwix for reading on a phone? My thinking is that if a few hundred LWers/EAs have this offline on their phones, that could go quite a long way. Burying phones with it on could also be good as a low hanging fruit (ideally you need a way of reading the data to be stored with the data). Happy to fund this if anyone wants to do it.

Greg C Jun 9, 2022, 11:33 AM
5 points
in reply to: P.’s comment on: Has anyone actually tried to convince Terry Tao or other top mathematicians to work on alignment?
No I mean links to him in person to talk to him (or for that matter, even an email address or any way of contacting him..).

Greg C Jun 9, 2022, 10:06 AM
4 points
in reply to: SP’s comment on: Has anyone actually tried to convince Terry Tao or other top mathematicians to work on alignment?
Oh wow, didn’t realise how recent the Huawei recruitment of Field medalists was! This from today. Maybe we need to convince Huawei to care about AGI Alignment :)

Greg C Jun 9, 2022, 9:49 AM
1 point
in reply to: Greg C’s comment on: Has anyone actually tried to convince Terry Tao or other top mathematicians to work on alignment?
Should also say—good that you are thinking about it P., and thanks for a couple of the links which I hadn’t seen before.

Greg C 9 Jun 2022 8:58 UTC
5 points
in reply to: Greg C’s comment on: Has anyone actually tried to convince Terry Tao or other top mathematicians to work on alignment?
Maybe reaching Demis Hassabis first is the way to go though, given that he’s already thinking about it, and has already mentioned it to Tao (according to the podcast). Does anyone have links to Demis? Would be good to know more about his “Avengers assemble” plan! The main thing is that the assembly needs to happen asap, at least for an initial meeting and “priming of the pump” as it were.

Greg C 9 Jun 2022 8:55 UTC
7 points
in reply to: Shmi’s comment on: Has anyone actually tried to convince Terry Tao or other top mathematicians to work on alignment?
Yes, I think the email needs to come from someone with a lot of clout (e.g. a top academic, or a charismatic billionaire; or even a high-ranking government official) if we actually want him to read it and take it seriously.

Greg C 8 Jun 2022 15:37 UTC
10 points
on: What have been the major “triumphs” in the field of AI over the last ten years?
Here’s a list that’s mostly from just the last few months (that is pretty scary): Deepmind’s Gato, Chinchilla, Flamingo and AlphaCode; Google’s Pathways, PaLM, SayCan, Socratic Models and TPUs; OpenAI’s DALL-E 2; EfficientZero; Cerebras

Greg C 28 Apr 2022 15:31 UTC
3 points
in reply to: lsusr’s comment on: [Advertisement] Want to hire me?
Interested in how you would go about throwing money at scalable altruistic projects. There is a lot of money and ideas around in EA, but a relative shortage of founders, I think.