The document Amodei wrote to explain his views on scaling to his colleagues — titled “The Big Blob of Compute Hypothesis” — became a classic among early OpenAI researchers. It described, in florid specifics, why Amodei thought that most AI researchers were being too clever. They would engineer all kinds of complicated solutions to whatever problems their models were having, when a much simpler solution — just make the thing bigger! — would work better.
Roose goes on to describe part of what is in the latter part of the document.
Amodei also thought that AI safety researchers suffered from micromanagement. Some researchers in his orbit, like the ones at Eliezer Yudkowsky’s MIRI (more recently in the news for other reasons) were arguing for a cybersecurity-like approach to AI safety, engineering complicated safety protocols to prevent models from going off the rails. Amodei disagreed. He thought that a better way to create safe AI was to get the high-level conditions right, and let models figure out on their own what was safe or unsafe.
I was really looking forward to reading this! Unfortunately, he declined to publish the complete document.
I am publishing only the first 10, since the rest is very dense and technical and consists mainly of Amodei arguing with various other AI safety researchers about why their suggested approaches are wrong.)
This is extremely unfortunate. If anyone knows Roose, I think it would be worth seeing if he would be willing to publish the remainder.
It occurs to me that if I only felt comfortable publishing part of a document, but I did not think it would be acceptable to do so, the course of action that I would be left with is to not publish any of it.
Publishing a partial document is, flatly, not unjournalistic behavior?
Meta note: this was at +19 karma when I read it. I’m 80% sure LessWrong becomes uncalibrated and entitled when discussing journalistic ethics. It’s a bad look with bad consequences, because LessWrong acts its most irrational on exactly the topics where journalists are most likely to see and notice it.
I mean, I think it’s good when people publish all the data associated with some piece of research, and it feels similar that I think journalists should also generally publish unedited interviews and such somewhere, to help the public audit the work and inspect what happened.
Seems strange to not include it, and needlessly frustrating to redact information of the public interest.
I agree publishing partial documents isn’t unjournalistic, and that LessWrong doesn’t seem to be the most calibrated when it comes to journalism-related takes, but I also don’t really understand Roose’s reasoning for only publishing part of the document. His reasons for publishing what he did are:
It’s an important document for historical reasons
It helps give insight into Amodei
To give an example of the type of juicy details you can get from his new book
To me, it seems like (1) and (2) would favor publishing the entire thing, especially (1). If you think its important to preserve these types of documents for history and that there is a strong public interest in people being able to read and study them, surely that would apply to the entire document? And the reasoning that the full document is long, dense, and technical doesn’t seem to counteract that. I’d guess that lots of historical records that have been of immense value in hindsight to our current understanding of history were some combination of those things, shouldn’t part of the point of preserving documents for history be that we may not know ahead of time exactly what the historians of the future will find to be relevant?
He thought that a better way to create safe AI was to get the high-level conditions right, and let models figure out on their own what was safe or unsafe.
So, what are the high-level conditions for the AIs to figure out friendliness on their own?
Kevin Roose recently published the first 10 pages of Dario Amodei’s “Big Blob of Compute” essay on his substack.
Roose goes on to describe part of what is in the latter part of the document.
I was really looking forward to reading this! Unfortunately, he declined to publish the complete document.
This is extremely unfortunate. If anyone knows Roose, I think it would be worth seeing if he would be willing to publish the remainder.
It occurs to me that if I only felt comfortable publishing part of a document, but I did not think it would be acceptable to do so, the course of action that I would be left with is to not publish any of it.
wtf Kevin? this is unjournalistic behavior
Publishing a partial document is, flatly, not unjournalistic behavior?
Meta note: this was at +19 karma when I read it. I’m 80% sure LessWrong becomes uncalibrated and entitled when discussing journalistic ethics. It’s a bad look with bad consequences, because LessWrong acts its most irrational on exactly the topics where journalists are most likely to see and notice it.
I mean, I think it’s good when people publish all the data associated with some piece of research, and it feels similar that I think journalists should also generally publish unedited interviews and such somewhere, to help the public audit the work and inspect what happened.
Seems strange to not include it, and needlessly frustrating to redact information of the public interest.
Copenhagen Interpretation of Journalism
I agree publishing partial documents isn’t unjournalistic, and that LessWrong doesn’t seem to be the most calibrated when it comes to journalism-related takes, but I also don’t really understand Roose’s reasoning for only publishing part of the document. His reasons for publishing what he did are:
It’s an important document for historical reasons
It helps give insight into Amodei
To give an example of the type of juicy details you can get from his new book
To me, it seems like (1) and (2) would favor publishing the entire thing, especially (1). If you think its important to preserve these types of documents for history and that there is a strong public interest in people being able to read and study them, surely that would apply to the entire document? And the reasoning that the full document is long, dense, and technical doesn’t seem to counteract that. I’d guess that lots of historical records that have been of immense value in hindsight to our current understanding of history were some combination of those things, shouldn’t part of the point of preserving documents for history be that we may not know ahead of time exactly what the historians of the future will find to be relevant?
Is this comparing his behavior to journalism as it is, or to what you wish it were?
Did he disclose that his fiancé works at Anthropic?
That’s not Kevin Roose, that’s his co-host, Casey Netwon.
Oh right, oops
Perhaps you could post the ask to contact Kevin to EA Forum, maybe someone there can get in contact with him?
So, what are the high-level conditions for the AIs to figure out friendliness on their own?