Incorrect: OpenAI is not aware of the risks of race dynamics.
OpenAI’s Charter contains the following merge-and-assist clause: “We are concerned about late-stage AGI development becoming a competitive race without time for adequate safety precautions. Therefore, if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project. We will work out specifics in case-by-case agreements, but a typical triggering condition might be “a better-than-even chance of success in the next two years.””
Being worried about race dynamics and then stopping at the last minute makes sense and seems a lot better than nothing. But I’m confused why this understanding doesn’t propagate to other beliefs/actions.
Specifically, below are some confusions I have with OpenAI’s worldview. If answered, these could give me a lot more hope in OpenAI’s direction.
On my models, we want to know as much about alignment as possible before we get close to AGI, and so it is incredibly important to have as much time as possible before we are close to AGI. I would much rather live in the world where we had 20 years to solve AI alignment than the world where we only have 10.
Again, according to my current worldview, actions such as releasing GPT-3 are extremely negative, because it tells everyone that LLMs work and thus accelerates capabilities and therefore also shortens timelines.
If the purpose of the merge-and-assist clause is to prevent a race dynamic, then it’s sufficient for that clause to trigger when OpenAI would otherwise decide to start racing. They can interpret their own decision-making, right? Right?
Being worried about race dynamics and then stopping at the last minute makes sense and seems a lot better than nothing. But I’m confused why this understanding doesn’t propagate to other beliefs/actions.
Specifically, below are some confusions I have with OpenAI’s worldview. If answered, these could give me a lot more hope in OpenAI’s direction.
How will you know that AGI has a >50% chance of success in the next two years? MIRI certainly seems to think this is hard.
How does OpenAI leadership feel about accelerating timelines? [1]
What are OpenAI leadership’s timelines right now? What are these timelines based off of?
Does OpenAI retroactively think that publishing that GPT-3 worked was a mistake? [2][3]
Will OpenAI publish GPT-4? what are the factors driving this decision?
On my models, we want to know as much about alignment as possible before we get close to AGI, and so it is incredibly important to have as much time as possible before we are close to AGI. I would much rather live in the world where we had 20 years to solve AI alignment than the world where we only have 10.
If the benefits are using GPT-3 to do alignment research, why not give it to just alignment researchers, and not tell anyone else?
Again, according to my current worldview, actions such as releasing GPT-3 are extremely negative, because it tells everyone that LLMs work and thus accelerates capabilities and therefore also shortens timelines.
If the purpose of the merge-and-assist clause is to prevent a race dynamic, then it’s sufficient for that clause to trigger when OpenAI would otherwise decide to start racing. They can interpret their own decision-making, right? Right?