This reads to me as obvious self-deception. Did you not make an agreement with the staffer who hired you that you would work in OpenAI’s interests while there? Do you not each day set the implicit expectation with your colleagues that you’re on the same team?
I reflected years ago that all contracts come with a hidden, secret clause, which is that “I am not trying to screw you over with this agreement”. You can be the sort of player who doesn’t have that, but this means I don’t want to make most deals with you, because I would have to put I so much extra work to make sure you’re not screwing me over.
Telling someone you’re secretly screwing them over for their own good… is not an honest or honorable way of interfacing with someone, and should not be the norm, including for people who you have severe disagreements with.
When hired by an employer, we agree to do certain work in exchange for compensation, not to optimize for the employer’s interests or what the CEO thinks the employer’s interests are. The implicit expectation with my colleagues is that I’m on their team, not necessarily the company’s. I work in my employer’s interests because I care about maximizing impact, because I take pride in my work, and because I explicitly told my manager I would finish a certain project this week.
In my view the implicit expectation you have of people by default is fairly weak, and signing a contract doesn’t change this much. In fact, the point of a contract is to make obligations explicit so we don’t have to rely on implicit trust.
When hired by an employer, we agree to do certain work in exchange for compensation, not to optimize for the employer’s interests or what the CEO thinks the employer’s interests are.
Actually it’s common for great companies to have visions that the people believe in that they make part of the hiring and onboarding process, and explicitly label and talk through (e.g. SpaceX’s “we’re going to Mars” or Stripe’s “Increase the GDP of the internet”). I think this is good, and I strongly expect that this is part of the culture Altman has set at OpenAI, so I expect it is much more of an implicit agreement there than it is if you (say) work at a restaurant as a waiter.
There are many equilibriums about how much people expect others to believe in the company that they work at. I guess I am coming at this from a culture where people work at a place because they believe in it, and I think this is a better equilibrium.
It’s something of an empirical question how good the existing companies are and how feasible it is to only work at places where you believe you’re improving the world. But it does seem to me that, if you’re at OpenAI but think it’s harmful to the world, you can just leave and make decent money elsewhere, I don’t think anyone is particularly trapped at the job there.
I guess I see these visions more as things companies try to filter for, inculcate, and perhaps require of executives, rather than ideologies that a rank-and-file engineer is ethically required to adopt. Maybe Lightcone and SpaceX are exceptions, but employees at most companies have a variety of reasons for working there. I’d guess the most common motivation for AI engineers is money. Is it dishonorable for a cracked IC at OpenAI to take a promotion to manager where they’re less effective?
Ok, what if they are motivated by OpenAI’s stated mission: “to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity”? It doesn’t say you should defer to Sam Altman and act as if endowing GPT-5.5 with the capability to spy on Americans benefits all of humanity. While I don’t agree with everything in the OP, it seems perfectly reasonable for an OpenAI employee who wants to benefit all of humanity to take protest actions, including slacking off at work and focusing on office politics if this is better than quitting. Why not just leave? Well, you could become a whistleblower, or the office politics could pay off and let you influence OpenAI for the better.
OpenAI leadership broke that implicit contract first. It was originally supposed to be a philanthropic thing for the benefit of humanity. It was supposed to be “open”. Then it became for-profit, now it’s going to work on killer robots for the military. To whatever extent there’s an implicit contract like you describe, it would also apply to the work that people previously did for it under false pretenses!
That seems like a legit point! It’s just very important to include information like that in your post when you’re proposing corporate sabotage. In political and murky domains where there’s so much noise and adversarial spreading of false narratives, it’s really scary to see posts that just say “I’m sure we all agree <thing> is bad, now let us do underhanded and dishonorable things to attack <thing>”. You can write that independent of whether something is good or bad, and instead write it whenever something is unpopular. It’s both more truth-tracking and less scary to see someone write “I encourage you to do underhanded things to <thing> because of <list of underhanded things that we know they did>.”
You open the post with a list of things that, while bad, are at best reason to quit and protest the company, not reasons to be dishonorable. This section, as far as I can tell, was about as much as you spent actually justifying corporate sabotage:
There is more open debate than I thought ther ewould be, at least in this part of Twitter, about whether we should prefer a democratically elected government or unelected private companies to have more power. I guess this is something people disagree on, but…I don’t. This seems like an important area for more discussion.
Let’s be clear: this was not about Anthropic telling the US military not to work on autonomous weapons on its own. Altman is advocating for the government being able to require private companies (and their employees) to provide whatever services it wants, even if they don’t currently do that thing. I know the term “fascism” has been thrown around a lot, but that is Actual Fascism. Here are some other ways to use that argument:
“Why should Sam Altman decide what should be done with that billion dollars instead of the government, which reflects the will of the people?”
“Why should a private citizen get to decide they don’t want to spy on their neighbors and report any hidden jews? That should be the decision of the government, which reflects the will of the people!”
This jump to ‘fascism’ is just cheap. Are you aware that Altman has repeatedly and publicly stated:
In my conversations over the weekend, I reiterated that Anthropci should not be designated as a SCR, and that we hope the DOW offers them the same terms we’ve agreed to.
Insofar as him endorsing the government’s threat against a private company is the ‘fascism’ you’re accusing, he has spoken out against it. You may wish to make some argument about why these words are not representative of the actions he will take, but instead you decided it was fine to label someone ‘Actual Fascism’ with capitals. Please hold yourself to higher standards than this.
The biggest problem that I have with the post is that quitting your job is considered a bigger deal than giving up on being an honorable person. Seems very far out from what I consider good behavior. I myself have gone and protested outside of OpenAI due to them racing to develop AGI while being well aware that this poses an extinction-level threat to humanity, so I know is quite possible to oppose a company without acting dishonorably in the process. If you work at OpenAI and no longer believe in the ethics of the company, you can just do the decent thing and quit.
I’m sure it’s possible to write a better version of this post. I hope someone does. Believe it or not, my specialty is engineering, not rhetoric.
My assessment of Sam Altman is that he’s a very good actor, very untrustworthy, and a nihilistic power-seeker who cares very little about benefit or harm to humanity as a whole. I agree that this post alone is only weak support for that assessment. A proper “compendium of reasons not to trust Sam Altman” would probably end up being a considerably longer post.
This reads to me as obvious self-deception. Did you not make an agreement with the staffer who hired you that you would work in OpenAI’s interests while there? Do you not each day set the implicit expectation with your colleagues that you’re on the same team?
I reflected years ago that all contracts come with a hidden, secret clause, which is that “I am not trying to screw you over with this agreement”. You can be the sort of player who doesn’t have that, but this means I don’t want to make most deals with you, because I would have to put I so much extra work to make sure you’re not screwing me over.
Telling someone you’re secretly screwing them over for their own good… is not an honest or honorable way of interfacing with someone, and should not be the norm, including for people who you have severe disagreements with.
When hired by an employer, we agree to do certain work in exchange for compensation, not to optimize for the employer’s interests or what the CEO thinks the employer’s interests are. The implicit expectation with my colleagues is that I’m on their team, not necessarily the company’s. I work in my employer’s interests because I care about maximizing impact, because I take pride in my work, and because I explicitly told my manager I would finish a certain project this week.
In my view the implicit expectation you have of people by default is fairly weak, and signing a contract doesn’t change this much. In fact, the point of a contract is to make obligations explicit so we don’t have to rely on implicit trust.
Actually it’s common for great companies to have visions that the people believe in that they make part of the hiring and onboarding process, and explicitly label and talk through (e.g. SpaceX’s “we’re going to Mars” or Stripe’s “Increase the GDP of the internet”). I think this is good, and I strongly expect that this is part of the culture Altman has set at OpenAI, so I expect it is much more of an implicit agreement there than it is if you (say) work at a restaurant as a waiter.
There are many equilibriums about how much people expect others to believe in the company that they work at. I guess I am coming at this from a culture where people work at a place because they believe in it, and I think this is a better equilibrium.
It’s something of an empirical question how good the existing companies are and how feasible it is to only work at places where you believe you’re improving the world. But it does seem to me that, if you’re at OpenAI but think it’s harmful to the world, you can just leave and make decent money elsewhere, I don’t think anyone is particularly trapped at the job there.
I guess I see these visions more as things companies try to filter for, inculcate, and perhaps require of executives, rather than ideologies that a rank-and-file engineer is ethically required to adopt. Maybe Lightcone and SpaceX are exceptions, but employees at most companies have a variety of reasons for working there. I’d guess the most common motivation for AI engineers is money. Is it dishonorable for a cracked IC at OpenAI to take a promotion to manager where they’re less effective?
Ok, what if they are motivated by OpenAI’s stated mission: “to ensure that artificial general intelligence—AI systems that are generally smarter than humans—benefits all of humanity”? It doesn’t say you should defer to Sam Altman and act as if endowing GPT-5.5 with the capability to spy on Americans benefits all of humanity. While I don’t agree with everything in the OP, it seems perfectly reasonable for an OpenAI employee who wants to benefit all of humanity to take protest actions, including slacking off at work and focusing on office politics if this is better than quitting. Why not just leave? Well, you could become a whistleblower, or the office politics could pay off and let you influence OpenAI for the better.
OpenAI leadership broke that implicit contract first. It was originally supposed to be a philanthropic thing for the benefit of humanity. It was supposed to be “open”. Then it became for-profit, now it’s going to work on killer robots for the military. To whatever extent there’s an implicit contract like you describe, it would also apply to the work that people previously did for it under false pretenses!
That seems like a legit point! It’s just very important to include information like that in your post when you’re proposing corporate sabotage. In political and murky domains where there’s so much noise and adversarial spreading of false narratives, it’s really scary to see posts that just say “I’m sure we all agree <thing> is bad, now let us do underhanded and dishonorable things to attack <thing>”. You can write that independent of whether something is good or bad, and instead write it whenever something is unpopular. It’s both more truth-tracking and less scary to see someone write “I encourage you to do underhanded things to <thing> because of <list of underhanded things that we know they did>.”
You open the post with a list of things that, while bad, are at best reason to quit and protest the company, not reasons to be dishonorable. This section, as far as I can tell, was about as much as you spent actually justifying corporate sabotage:
This jump to ‘fascism’ is just cheap. Are you aware that Altman has repeatedly and publicly stated:
Insofar as him endorsing the government’s threat against a private company is the ‘fascism’ you’re accusing, he has spoken out against it. You may wish to make some argument about why these words are not representative of the actions he will take, but instead you decided it was fine to label someone ‘Actual Fascism’ with capitals. Please hold yourself to higher standards than this.
The biggest problem that I have with the post is that quitting your job is considered a bigger deal than giving up on being an honorable person. Seems very far out from what I consider good behavior. I myself have gone and protested outside of OpenAI due to them racing to develop AGI while being well aware that this poses an extinction-level threat to humanity, so I know is quite possible to oppose a company without acting dishonorably in the process. If you work at OpenAI and no longer believe in the ethics of the company, you can just do the decent thing and quit.
I’m sure it’s possible to write a better version of this post. I hope someone does. Believe it or not, my specialty is engineering, not rhetoric.
My assessment of Sam Altman is that he’s a very good actor, very untrustworthy, and a nihilistic power-seeker who cares very little about benefit or harm to humanity as a whole. I agree that this post alone is only weak support for that assessment. A proper “compendium of reasons not to trust Sam Altman” would probably end up being a considerably longer post.