Excuse me for the half-off-topic, but would you give me visibility into what might get you to work at an EA/rationality org?
Or in other phrasings:
What is preventing it?
What could change that would get you to move to such an org? Feel free to ask for magical stuff that would never happen in real life, I’m in an early “learn about the users” stage here
Thanks!
(Context TL;DR: Not enough software engineers apply to such orgs, and the orgs have no idea what’s going on since our community theoretically has so many developers. I’m trying to help figure it out)
Sure, I think it’s pretty simple in my case. I’m able to make a lot of money. I guess I shouldn’t say exactly how much, but take a look at level.fyi and look at positions equivalent to E7/L7 and that tells you something about the kind of compensation I command. Given I work at a startup (so no liquidity for my stock yet) that’s growing healthily I have a large upside with good expected value of payout. That can translate into a lot of targeted giving at funding opportunities I’m interested in prioritizing: neglected things in AI safety that are too risky/uncertain to get much other funding but that I think are interesting and deserve a shot.
To get me to apply it’d have to be clear that the work I’m doing is worth in excess of $1mm a year, otherwise I think I’m better off staying in industry and donating to fund others (since my earning potential is continuing to grow over time; if I ever top out then that might change the calculus slightly). Also, I’m going to have to be paid enough to fund the life I like living, which exists off the chunk of money I don’t donate. So in current money we’re looking at cash compensation around $300k (which also accounts for money I donate to non-EA things) and then ~$700k a year in relevant excess value generated or donation offsets to account for the difference in compensation from the alternative.
I think there’s also for me got to be some story about career trajectory. Because I’ve not topped out in my career, the role has to have a clear story about how it’s going to help me grow since I currently have the possibility of making 2-5x more than I make now in the next several years doing what I do now if I can do it a bit better. If the story is going to be that I take a few years off to work at an EA org to do good that would be fine except I’m passing up a lot of lifetime compensation in expectation, which also means passing up a lot of potential donations.
One final bit to figure in: I tried my hand a bunch of AI safety research. I produced what I think is some interesting stuff, it’s gained a little traction and others are doing things with it, but at the end of the day I like building more than anything else. So it also has to be that the job being offered is a similar software engineering job that I’ll like doing day to day. There is some value in being overqualified for a job and being able to execute on it really well, but if the org has fewer than 50-100 engineers it’s not clear there’s a lot of value for me to uniquely bring to it. Even as CTO I’m not sure there’s enough there there unless this just happens to be an EA-aligned startup with high growth potential.
If an EA org would tell you that you’d produce more value for the world by working directly for them, building things, while also earning a salary (insert some number here) that lets you live well and donate to non-EA causes, even though the EA org has fewer than 50 engineers, compared to having you donate as much as you’d maximally donate ($1M/year?), would you move to the EA org?
(Reminder: I’m asking this as a “debug” question, partly to check my understanding. Pushbacks are welcome)
[Answering for myself only, though I’m in a similar position—was Principal Engineer at a FAANG for many years, recently moved to a late-stage startup in a VP-level IC role, with comp in the range that Gordon describes].
I think it’d take a lot to convince me that my comparative advantage in … the stuff described in the post (strategic tech decisions, with details down to coding and the tens of thousands of dumb little decisions that make a software product, and bridging the trust required by individual engineers and senior management) … is more valuable in an EA/Rationality org than a commercial software company.
Without some amount of market discipline and medium-term customer success indicators (denominated in dollars), I don’t think I can have the confidence and good instincts about what to change (and what to drop entirely) that make me valuable.
This is related to the discussion about how to spend EA money well—it’s not that it’s impossible to hire people, it’s that there aren’t enough well-understood projects to justify hiring expensive people. It’s not QUITE “you can’t afford me”, it’s more “you don’t have a model where you need me enough to hire me”.
Regarding “you don’t have a model where you need me enough to hire me”—may I ask if that’s something that an EA org (or several such orgs) have told you, or something you’re guessing?
Purely guessing. I haven’t seen any job postings or other indications that they have a large and confused enough software engineering organization to make good use of (what I think is my) main strengths.
I’ve recently changed jobs, so not looking for another change immediately, but I’d be happy to chat with execs of orgs that wonder if they need someone like me—generally that would be engineering orgs of 120-500, usually in 10-40 teams with multiple products and somewhat unaligned (heh) vision.
lol I resonate with the desire to get into a confused software engineering org and help bring order :) Though I am not an experienced manager like yourself
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100+ devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Maybe? It’d at least make it an option, but I think Dagon’s reply is right: it’d be very hard for me to evaluate if you actually need me, and if you don’t actually need me then the opportunity cost for everyone ends up being high enough that it’s probably a no.
it’d be very hard for me to evaluate if you actually need me
May I ask if you evaluated this yourself? (I’m guessing: based on EA orgs being too small and with too little growth, so your own edge isn’t there, so they better hire someone else. But I’d be happy for clarity here if you’d help me out)
Yes, although not recently for any specific position. If someone came to me with a position or I saw an ad for a position (closest thing recently had been Lightcone, and I think I didn’t pursue basically for reasons above based on what I read) Is have to think about it.
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100 devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Sorry I missed there was a question here originally, thus the late reply.
I guess my thought is put your money where your mouth is. That is, if EA orgs really want more senior folks, they’re going to have to pay for it and not rely on people being mission aligned enough to take a pay cut.
I think realistically the way to get folks is to offer more cash than they can get in the private sector (where stock is often used to offset cash compensation). So for example offer 30% more cash than 95th percentile for the level you want to hire. For example, if someone wanted to hire me to work at an EA org they’d have to pay me at least $450k/year annual cash comp by this algorithm, and thinking about it being offered that much cash given the tradeoffs of working at a smaller org where I won’t get to exercise all my skills feels fair in that I’d be even on taking that option vs. continuing at my current job.
I haven’t closely followed EA developer job postings in years, but my sense is that they fall into three categories:
WebDev
CRUD apps
ML Eng
Any of these would be a waste of a staff engineer- the first two because what the jobs require is such a small subset of what the engineer can do, and the third because it requires a bunch of skills they don’t have (while still not using the ones they do).
From a total utility perspective, it seems better that EA roles are filled by junior engineers, while senior engineers leverage their additional skills to earn more.
This doesn’t apply to absolutely every job (LW/EAF are serious works of engineering), but I do wonder if EA orgs are looking to hire more impressive people than they can actually make use of.
Startups sometimes have founders or early employees who are staff (or higher) engineers.
Sometimes this goes terribly: the staff engineer is used to working in a giant bureaucracy, so instead of writing code they organize a long series of meetings to produce a UML diagram or something, and the company fails.
Sometimes this goes amazingly: the staff engineer can fix bugs 10x faster than the competitors’ junior engineers while simultaneously having the soft skills to talk to customers, interview users, etc.
If you are in the former category, EA organizations mostly don’t need you. If you are in the latter category though, EA organizations desperately need you, for the same reasons startups want to hire you, even though you also have skills that we won’t be able to use.
If someone is considering applying to CEA I’m happy to talk with them about whether it would be a good fit for both sides, including which of their skills would be useful, which new ones they would have to learn, and how they would feel about that. Some people really like being able to meet a promising EA at a party, hear about one of their issues, rapidly bang out a PR to fix it, and then immediately see that EA become more impactful (and know that without them—it wouldn’t happen). But other people really like the “make a UML diagram” style of work, and don’t enjoy working here. It’s hard for me to guess or to give a generic answer that will fit everyone.
It seems like you are bucketing senior eng skills into bureaucracy or… agility? ability to work quickly and responsively? That is missing most of what makes staff engineers staff engineers. Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later. EA is growing but AFAIK not at nearly the speed it would take to make use of those skills: you’re more likely to end up with something terribly overbuilt for the purpose.
From what you describe, I think you’d be much better off looking for a kick-ass mid-level PM with a coding background who wants to get back into engineering. They’d be giving up much less (both in terms of money, painstakingly acquired skills they enjoy using, and future option value), and are more likely to have the skills you actually want.
> Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later.
I’m sure this is true of some startups, but was not true of mine, nor the ones I was thinking of what I wrote that.
Senior engineers are like… Really good engineers? Not sure how to describe it in a non-tautological way. I somewhat regularly see a senior engineer solve in an afternoon a problem which a junior engineer has struggled with for weeks.
Being able to move that quickly is extremely valuable for startups.
(I agree that many staff engineers are not “really good engineers” in the way I am describing, and are therefore probably not of interest to many EA organizations.)
I’m a step lower here (senior engineer) and the reasons I and the people I know mostly don’t apply to EA orgs is that we get signals that they don’t want or value us: No remote work, generally not many actual engineer openings, various posts saying the way to get a job is to work unpaid for months in the hopes of getting attention, and at least one EA org lowballs salaries.
Excuse me for the half-off-topic, but would you give me visibility into what might get you to work at an EA/rationality org?
Or in other phrasings:
What is preventing it?
What could change that would get you to move to such an org? Feel free to ask for magical stuff that would never happen in real life, I’m in an early “learn about the users” stage here
Thanks!
(Context TL;DR: Not enough software engineers apply to such orgs, and the orgs have no idea what’s going on since our community theoretically has so many developers. I’m trying to help figure it out)
Sure, I think it’s pretty simple in my case. I’m able to make a lot of money. I guess I shouldn’t say exactly how much, but take a look at level.fyi and look at positions equivalent to E7/L7 and that tells you something about the kind of compensation I command. Given I work at a startup (so no liquidity for my stock yet) that’s growing healthily I have a large upside with good expected value of payout. That can translate into a lot of targeted giving at funding opportunities I’m interested in prioritizing: neglected things in AI safety that are too risky/uncertain to get much other funding but that I think are interesting and deserve a shot.
To get me to apply it’d have to be clear that the work I’m doing is worth in excess of $1mm a year, otherwise I think I’m better off staying in industry and donating to fund others (since my earning potential is continuing to grow over time; if I ever top out then that might change the calculus slightly). Also, I’m going to have to be paid enough to fund the life I like living, which exists off the chunk of money I don’t donate. So in current money we’re looking at cash compensation around $300k (which also accounts for money I donate to non-EA things) and then ~$700k a year in relevant excess value generated or donation offsets to account for the difference in compensation from the alternative.
I think there’s also for me got to be some story about career trajectory. Because I’ve not topped out in my career, the role has to have a clear story about how it’s going to help me grow since I currently have the possibility of making 2-5x more than I make now in the next several years doing what I do now if I can do it a bit better. If the story is going to be that I take a few years off to work at an EA org to do good that would be fine except I’m passing up a lot of lifetime compensation in expectation, which also means passing up a lot of potential donations.
One final bit to figure in: I tried my hand a bunch of AI safety research. I produced what I think is some interesting stuff, it’s gained a little traction and others are doing things with it, but at the end of the day I like building more than anything else. So it also has to be that the job being offered is a similar software engineering job that I’ll like doing day to day. There is some value in being overqualified for a job and being able to execute on it really well, but if the org has fewer than 50-100 engineers it’s not clear there’s a lot of value for me to uniquely bring to it. Even as CTO I’m not sure there’s enough there there unless this just happens to be an EA-aligned startup with high growth potential.
Thank you very much for the high-detail answer <3
If I may continue debugging:
If an EA org would tell you that you’d produce more value for the world by working directly for them, building things, while also earning a salary (insert some number here) that lets you live well and donate to non-EA causes, even though the EA org has fewer than 50 engineers, compared to having you donate as much as you’d maximally donate ($1M/year?), would you move to the EA org?
(Reminder: I’m asking this as a “debug” question, partly to check my understanding. Pushbacks are welcome)
[Answering for myself only, though I’m in a similar position—was Principal Engineer at a FAANG for many years, recently moved to a late-stage startup in a VP-level IC role, with comp in the range that Gordon describes].
I think it’d take a lot to convince me that my comparative advantage in … the stuff described in the post (strategic tech decisions, with details down to coding and the tens of thousands of dumb little decisions that make a software product, and bridging the trust required by individual engineers and senior management) … is more valuable in an EA/Rationality org than a commercial software company.
Without some amount of market discipline and medium-term customer success indicators (denominated in dollars), I don’t think I can have the confidence and good instincts about what to change (and what to drop entirely) that make me valuable.
This is related to the discussion about how to spend EA money well—it’s not that it’s impossible to hire people, it’s that there aren’t enough well-understood projects to justify hiring expensive people. It’s not QUITE “you can’t afford me”, it’s more “you don’t have a model where you need me enough to hire me”.
(Thank you for joining in! <3)
Regarding “you don’t have a model where you need me enough to hire me”—may I ask if that’s something that an EA org (or several such orgs) have told you, or something you’re guessing?
Purely guessing. I haven’t seen any job postings or other indications that they have a large and confused enough software engineering organization to make good use of (what I think is my) main strengths.
I’ve recently changed jobs, so not looking for another change immediately, but I’d be happy to chat with execs of orgs that wonder if they need someone like me—generally that would be engineering orgs of 120-500, usually in 10-40 teams with multiple products and somewhat unaligned (heh) vision.
lol I resonate with the desire to get into a confused software engineering org and help bring order :) Though I am not an experienced manager like yourself
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100+ devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Maybe? It’d at least make it an option, but I think Dagon’s reply is right: it’d be very hard for me to evaluate if you actually need me, and if you don’t actually need me then the opportunity cost for everyone ends up being high enough that it’s probably a no.
May I ask if you evaluated this yourself? (I’m guessing: based on EA orgs being too small and with too little growth, so your own edge isn’t there, so they better hire someone else. But I’d be happy for clarity here if you’d help me out)
Yes, although not recently for any specific position. If someone came to me with a position or I saw an ad for a position (closest thing recently had been Lightcone, and I think I didn’t pursue basically for reasons above based on what I read) Is have to think about it.
I see, thanks
As I said in the other thread:
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100 devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Sorry I missed there was a question here originally, thus the late reply.
I guess my thought is put your money where your mouth is. That is, if EA orgs really want more senior folks, they’re going to have to pay for it and not rely on people being mission aligned enough to take a pay cut.
I think realistically the way to get folks is to offer more cash than they can get in the private sector (where stock is often used to offset cash compensation). So for example offer 30% more cash than 95th percentile for the level you want to hire. For example, if someone wanted to hire me to work at an EA org they’d have to pay me at least $450k/year annual cash comp by this algorithm, and thinking about it being offered that much cash given the tradeoffs of working at a smaller org where I won’t get to exercise all my skills feels fair in that I’d be even on taking that option vs. continuing at my current job.
I haven’t closely followed EA developer job postings in years, but my sense is that they fall into three categories:
WebDev
CRUD apps
ML Eng
Any of these would be a waste of a staff engineer- the first two because what the jobs require is such a small subset of what the engineer can do, and the third because it requires a bunch of skills they don’t have (while still not using the ones they do).
From a total utility perspective, it seems better that EA roles are filled by junior engineers, while senior engineers leverage their additional skills to earn more.
This doesn’t apply to absolutely every job (LW/EAF are serious works of engineering), but I do wonder if EA orgs are looking to hire more impressive people than they can actually make use of.
Startups sometimes have founders or early employees who are staff (or higher) engineers.
Sometimes this goes terribly: the staff engineer is used to working in a giant bureaucracy, so instead of writing code they organize a long series of meetings to produce a UML diagram or something, and the company fails.
Sometimes this goes amazingly: the staff engineer can fix bugs 10x faster than the competitors’ junior engineers while simultaneously having the soft skills to talk to customers, interview users, etc.
If you are in the former category, EA organizations mostly don’t need you. If you are in the latter category though, EA organizations desperately need you, for the same reasons startups want to hire you, even though you also have skills that we won’t be able to use.
If someone is considering applying to CEA I’m happy to talk with them about whether it would be a good fit for both sides, including which of their skills would be useful, which new ones they would have to learn, and how they would feel about that. Some people really like being able to meet a promising EA at a party, hear about one of their issues, rapidly bang out a PR to fix it, and then immediately see that EA become more impactful (and know that without them—it wouldn’t happen). But other people really like the “make a UML diagram” style of work, and don’t enjoy working here. It’s hard for me to guess or to give a generic answer that will fit everyone.
It seems like you are bucketing senior eng skills into bureaucracy or… agility? ability to work quickly and responsively? That is missing most of what makes staff engineers staff engineers. Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later. EA is growing but AFAIK not at nearly the speed it would take to make use of those skills: you’re more likely to end up with something terribly overbuilt for the purpose.
From what you describe, I think you’d be much better off looking for a kick-ass mid-level PM with a coding background who wants to get back into engineering. They’d be giving up much less (both in terms of money, painstakingly acquired skills they enjoy using, and future option value), and are more likely to have the skills you actually want.
> Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later.
I’m sure this is true of some startups, but was not true of mine, nor the ones I was thinking of what I wrote that.
Senior engineers are like… Really good engineers? Not sure how to describe it in a non-tautological way. I somewhat regularly see a senior engineer solve in an afternoon a problem which a junior engineer has struggled with for weeks.
Being able to move that quickly is extremely valuable for startups.
(I agree that many staff engineers are not “really good engineers” in the way I am describing, and are therefore probably not of interest to many EA organizations.)
I’m a step lower here (senior engineer) and the reasons I and the people I know mostly don’t apply to EA orgs is that we get signals that they don’t want or value us: No remote work, generally not many actual engineer openings, various posts saying the way to get a job is to work unpaid for months in the hopes of getting attention, and at least one EA org lowballs salaries.
Thank you very much for sharing! This comment is valuable for me