You say ‘We have no intention of doing any such thing. The company is perfectly capable of carrying on without Altman. We have every intention of continuing on OpenAI’s mission, led by the existing executive team. Altman promised to help with the transition in the board meeting. If he instead chooses to attempt to destroy OpenAI and its mission, that is his decision. It also proves he was incompatible with our mission and we needed to remove him.’
OpenAI’s charter seems consistent with Toner’s statement that “The destruction of the company could be consistent with the board’s mission.” Here are some quotes:
We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome.
...
We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project.
...
We will actively cooperate with other research and policy institutions; we seek to create a global community working together to address AGI’s global challenges.
Telling Toner to stay quiet about the charter seems like telling a fire captain to stay quiet about the fact that trainee firefighters may someday need to enter a burning building.
My feeling: It’s not Toner’s fault that she reminded people about the charter. It’s everyone else’s fault for staying quiet about it. It’s like if on the last day of firefighter training, one of the senior firefighters leading the training said “btw, being a firefighter sometimes means running into a burning building to save someone” and everyone was aghast—“you’re not supposed to mention that! you’re gonna upset the trainees and scare them away!”
The entire situation seems a little absurd to me. In my mind, effective firefighter training means psychologically preparing a trainee to enter a burning building from day one. (I actually made a comment about the value of pre-visualization in emergency situations about a year ago.) Maybe OpenAI execs should have been reviewing the charter with employees at every all-hands meeting, psychologically preparing them for the possibility that they might someday need to e.g. destroy the company. It feels unfair to blame Toner that things got to the point they did.
Telling Toner to stay quiet about the charter seems like telling a fire captain to stay quiet about the fact that trainee firefighters may someday need to enter a burning building.
Or perhaps it would be more analogous to firefighting tactics historically, which, in the absence of extremely large mechanical pumps & water sources, frequently forced resorts to making firebreaks, which can involve bans, deliberate destruction, controlled burns, or in the case of fighting fires in cities, razing* entire streets down to the ground (using dynamite if you must).
No one wants to dynamite thousands of innocent peoples’ homes, but if you are in the middle of the San Francisco earthquake and there is a city-wide fire raging and all the water pipes are busted (assuming you were lucky enough to be in a time & place that they ever existed)...
* if you like anime, the Short Peace anthology shows this in the “Combustible” short. It’s spectacularly animated. Or, by sheer coincidence, I’m watching Promare about sci-fi fire fighting mecha, and the protagonist uses a traditional Japanese firefighting outfit with a hooked spear-like weapon to fight fire-aliens… and the original hook existed, of course, not to duel fire-aliens but to help demolish buildings in cities (whether or not they are on fire yet).
I think this is a good analogy. Though I think “one day you might have to dynamite a bunch of innocent people’s homes to keep a fire from spreading, that’s part of the job” is a good thing to have in the training if that’s the sort of thing that’s likely to come up.
OpenAI’s charter seems consistent with Toner’s statement that “The destruction of the company could be consistent with the board’s mission.” Here are some quotes:
https://openai.com/charter
Telling Toner to stay quiet about the charter seems like telling a fire captain to stay quiet about the fact that trainee firefighters may someday need to enter a burning building.
My feeling: It’s not Toner’s fault that she reminded people about the charter. It’s everyone else’s fault for staying quiet about it. It’s like if on the last day of firefighter training, one of the senior firefighters leading the training said “btw, being a firefighter sometimes means running into a burning building to save someone” and everyone was aghast—“you’re not supposed to mention that! you’re gonna upset the trainees and scare them away!”
The entire situation seems a little absurd to me. In my mind, effective firefighter training means psychologically preparing a trainee to enter a burning building from day one. (I actually made a comment about the value of pre-visualization in emergency situations about a year ago.) Maybe OpenAI execs should have been reviewing the charter with employees at every all-hands meeting, psychologically preparing them for the possibility that they might someday need to e.g. destroy the company. It feels unfair to blame Toner that things got to the point they did.
Or perhaps it would be more analogous to firefighting tactics historically, which, in the absence of extremely large mechanical pumps & water sources, frequently forced resorts to making firebreaks, which can involve bans, deliberate destruction, controlled burns, or in the case of fighting fires in cities, razing* entire streets down to the ground (using dynamite if you must).
No one wants to dynamite thousands of innocent peoples’ homes, but if you are in the middle of the San Francisco earthquake and there is a city-wide fire raging and all the water pipes are busted (assuming you were lucky enough to be in a time & place that they ever existed)...
* if you like anime, the Short Peace anthology shows this in the “Combustible” short. It’s spectacularly animated. Or, by sheer coincidence, I’m watching Promare about sci-fi fire fighting mecha, and the protagonist uses a traditional Japanese firefighting outfit with a hooked spear-like weapon to fight fire-aliens… and the original hook existed, of course, not to duel fire-aliens but to help demolish buildings in cities (whether or not they are on fire yet).
I think this is a good analogy. Though I think “one day you might have to dynamite a bunch of innocent people’s homes to keep a fire from spreading, that’s part of the job” is a good thing to have in the training if that’s the sort of thing that’s likely to come up.