I kinda reject the energy of the hypothetical? But I can speak to some things I wish I saw OpenAI doing:
Having some internal sense amongst employees about whether they’re doing something “good” given the stakes, like Google’s old “don’t be evil” thing. Have a culture of thinking carefully about things and managers taking considerations seriously, rather than something more like management trying to extract as much engineering as quickly as possible without “drama” getting in the way.
(Perhaps they already have a culture like this! I haven’t worked there. But my prediction is that it is not, and the org has a more “extractive” relationship to its employees. I think that this is bad, causes working toward danger, and exacerbates bad outcomes.)
To the extent that they’re trying to have the best AGI tech in order to provide “leadership” of humanity and AI, I want to see them be less shady / marketing / spreading confusion about the stakes.
They worked to pervert the term “alignment” to be about whether you can extract more value from their LLMs, and distract from the idea that we might make digital minds that are copyable and improvable, while also large and hard to control. (While pushing directly on AGI designs that have the “large and hard to control” property, which I guess they’re denying is a mistake, but anyhow.)
I would like to see less things perverted/distracted/confused, like it’s according-to-me entirely possible for them to state more clearly what the end of all this is, and be more explicit about how they’re trying to lead the effort.
Reconcile with Anthropic. There is no reason, speaking on humanity’s behalf, to risk two different trajectories of giant LLMs built with subtly different technology, while dividing up the safety know-how amidst both organizations.
Furthermore, I think OpenAI kind-of stole/appropriated the scaling idea from the Anthropic founders, who left when they lost a political battle about the direction of the org. I suspect it was a huge fuck-you when OpenAI tried to spread this secret to the world, and continued to grow their org around it, while ousting the originators. If my model is at-all-accurate, I don’t like it, and OpenAI should look to regain “good standing” by acknowledging this (perhaps just privately), and looking to cooperate.
Idk, maybe it’s now legally impossible/untenable for the orgs to work together, given the investors or something? Or given mutual assumption of bad-faith? But in any case this seems really shitty.
I also mentioned some other things in this comment.
Out of interest—if you had total control over OpenAI—what would you want them to do?
I kinda reject the energy of the hypothetical? But I can speak to some things I wish I saw OpenAI doing:
Having some internal sense amongst employees about whether they’re doing something “good” given the stakes, like Google’s old “don’t be evil” thing. Have a culture of thinking carefully about things and managers taking considerations seriously, rather than something more like management trying to extract as much engineering as quickly as possible without “drama” getting in the way.
(Perhaps they already have a culture like this! I haven’t worked there. But my prediction is that it is not, and the org has a more “extractive” relationship to its employees. I think that this is bad, causes working toward danger, and exacerbates bad outcomes.)
To the extent that they’re trying to have the best AGI tech in order to provide “leadership” of humanity and AI, I want to see them be less shady / marketing / spreading confusion about the stakes.
They worked to pervert the term “alignment” to be about whether you can extract more value from their LLMs, and distract from the idea that we might make digital minds that are copyable and improvable, while also large and hard to control. (While pushing directly on AGI designs that have the “large and hard to control” property, which I guess they’re denying is a mistake, but anyhow.)
I would like to see less things perverted/distracted/confused, like it’s according-to-me entirely possible for them to state more clearly what the end of all this is, and be more explicit about how they’re trying to lead the effort.
Reconcile with Anthropic. There is no reason, speaking on humanity’s behalf, to risk two different trajectories of giant LLMs built with subtly different technology, while dividing up the safety know-how amidst both organizations.
Furthermore, I think OpenAI kind-of stole/appropriated the scaling idea from the Anthropic founders, who left when they lost a political battle about the direction of the org. I suspect it was a huge fuck-you when OpenAI tried to spread this secret to the world, and continued to grow their org around it, while ousting the originators. If my model is at-all-accurate, I don’t like it, and OpenAI should look to regain “good standing” by acknowledging this (perhaps just privately), and looking to cooperate.
Idk, maybe it’s now legally impossible/untenable for the orgs to work together, given the investors or something? Or given mutual assumption of bad-faith? But in any case this seems really shitty.
I also mentioned some other things in this comment.