OpenAI is no longer so open—we know almost nothing about GPT-4’s architecture.
Fantastic. This feels like a step in the right direction towards no longer letting just anyone use this to improve their capability research or stack their own capability research on top of it.
For reference, I’ve seen ChatGTP play chess, and while it played a very good opening, it became less and less reliable as the game went on and frequently lost track of the board.
That image so perfectly sums up how AI’s are nothing like us, in that the characters they present do not necessarily reflect their true values, that it needs to go viral.
Based on a few of his recent tweets, I’m hoping for a serious way to turn Elon Musk back in the direction he used to be facing and get him to publically go hard on the importance of the field of alignment. It’d be too much to hope for though to get him to actually fund any researchers, though. Maybe someone else.
At that level of power, I imagine that general intelligence will be a lot easier to create.
But not with something powerful enough to engineer nanotech.
With the strawberries thing, the point isn’t that it couldn’t do those things, but that it won’t want to. After making itself smart enough to engineer nanotech, it’s developing ‘mind’ will have run off in unintended directions and it will have wildly different goals that what we wanted it to have.
Quoting EY from this video: “the whole thing I’m saying is that we do not know how to get goals into a system.” <-- This is the entire thing that researchers are trying to figure out how to do.
They also recorded this follow-up with Yudkowsky if anyone’s interested:
https://twitter.com/BanklessHQ/status/1627757551529119744
______________
>Enrico Fermi was saying that fission chan reactions were 50 years off if they could ever be done at all, two years before he built the first nuclear pile. The Wright brothers were saying heavier-than-air flight was 50 years off shortly before they built the first Wright flyer.
The one hope we may be able to cling to is that this logic works in the other direction too—that AGI may be a lot closer than estimated, but so might alignment.
Here’s a dumb idea: if you have a misaligned AGI, can you keep it inside a box and have it teach you some things about alignment, perhaps through some creative lies?
I wouldn’t call it evil, but I would say that it’s playing with fire.
All good points. I suspect the best path into the future looks like: everyone’s optimistic and then a ‘survivable disaster’ happens with AI. Ideally, we’d want all the panic to happen in one big shot—it’s the best way to motivate real change.
>How do you address the “capabilities overhang” argument?
I don’t, I’m just not willing to bet the Earth that a world-ender will be proceed by mere mass murders. It might be, but let’s not risk it.
As I say, if there’s a 1% chance of doom, then we should treat it as nearly 100%.
This is why I’m crossing my fingers for a ‘survivable disaster’ - an AI that merely kills a lot of people instead of everyone. Maybe then people would take it seriously.
Coming up with a solution for spreading awareness of the problem is a difficult and important problem that ordinary people can actually tackle, and that’s what I want to try.
When did I say that raising awareness is all that we need to do?
Any ideas you have for overcoming the problems of alarmism would be good.
Pretty sobering, and it’s pretty clear that it’s beyond the time we got serious about this. I might put together a post of my own calling for any creative ideas that ordinary people can implement to help the cause, but the most obvious thing is to raise awareness. I hope Yudkowsky gets the chance to do a lot more interviews like this.
Yudkowsky, if you ever see this, please don’t give up hope. Crazy breakthroughs do happen, and more people are getting into alignment as time goes on.
Have to disagree with you on this point. I’m in the camp of “If there’s a 1% chance that AI doom is real, we should be treating it like a 99% chance.”