Becoming a Staff Engineer
I’m trying to think of things to write that would be both easy to hammer out and valuable enough to garner upvotes during these few days when upvotes are extra valuable. So how about a topic I know a lot about: my job!
I’m a senior staff software engineer. What does that mean?
There’s a couple ways to describe it.
Nominally it’s a software engineering job equivalent to being a director, which is common big-org management speak for a manager who manages other managers (but not generally someone who manages directors—that’s a VP). I often tell people I’m in “engineering middle management”, which is a funny way of saying that I’m expected to be highly autonomous, mostly identify my own projects and drive them to success, and operate over multiple teams (if my projects aren’t impacting multiple teams or groups of teams then that’s too small a project and I should delegate it to a staff or senior engineer if I think it’s important).
There’s different ways to do the job, but most of the time it means I don’t write code. Not because it’s beneath me but because spending time writing code is almost never high leverage enough for it to be the right choice (there’s almost always some other engineer who needs the growth opportunity of writing the code that I would be stealing from them if I wrote the code). This might be somewhat unique to me though because I seem to have higher comparative advantage in a few other skills: decision making (being accountable for making the right technical choices for the team and company), alignment building (getting everyone to agree on what we’re doing), project management (making sure everyone knows what to do and when they need to do it by), and mentorship and teaching (helping others learn skills that will help them grow or at least achieve what they want). I have to find more than zero time to write code to stay connected to the ground truth of the work, but it tends to be restricted to side quests that don’t have hard deadlines, hackathons, and Advent of Code.
Unfortunately, there’s no good easy way to describe the job. I didn’t understand it until I was already doing it, and I probably still don’t understand it. This seems to be an inherent challenge for both managers and staff engineers: fundamentally your job is to decide what needs to get done and then get it done, and you do whatever it is that you can best do to make that happen. The rest is just details about your skills, the expectations of your organization, and what your org needs, thus it’s somewhat hard to pin down what you do all day. For other perspectives, check out the StaffEng site.
That said, what if you want to become a staff engineer? How do you do it?
I can only tell you what I did. Hopefully it’s helpful.
First, I spent 8 years as a software engineer (and more like 25 years as a programmer). I got really really good at writing software. So good that I got bored. It’s not that there aren’t interesting technical problems to still solve, only that they don’t excite me the same way they used to because I have a deep, experience-based understanding of the work required to solve software problems, so there’s less for me to learn about the world through the process of doing the work. I still get excited about doing cool things with databases and distributed transaction processing (my primary area of specialization), but mostly I know that if I wanted to do them it would take a team and a couple years of work to make it happen, plus I’d have to find a business reason to justify the investment to do the cool work. So first step to being a staff software engineer is get so good at being a software engineering that you start to get bored of it.
Once I started to get bored I had to figure out what to do. I thought I wanted to go into management, mostly because I didn’t know about staff engineering at the time and figured it was the only option. So I went into management. First job: manage myself. My job became Head of SRE, which is a fancy way of saying I managed the Site Reliability Engineering function at a startup without any reports. This wasn’t too different from my previous job as a senior software engineer, except now I had a mandate—a scope of responsibility to do whatever needed to be done within it. I planned my own work in coordination with the CTO, instituted programs (like oncall training for the engineers), and for a while even had a direct report working half-time for me.
I was able to jump on this opportunity because the startup had grown enough that it made sense to manage our infrastructure in house rather than outsource it. We had been deployed on Heroku and our bill was high enough that I could justify my salary in cost savings if I could get us running smoothly on AWS directly, so I did that. Over the course of 9 months I used this project as a springboard to go from senior engineer to head of a function. All I had to do was deliver on the project successfully and ask for the new job (by providing a clear plan about what I was going to do with it). So, second step was to identify an opportunity to become more autonomous, prove I could handle the responsibility, and justify the need for the role.
Now, along the way I did a lot of things that made it possible for me to pull this transition off. For example, in my first two years at this startup before becoming Head of SRE I was also the scrum master (aka the project manager). I took this role on mostly because I had seen really bad project management at other companies, and I didn’t want something similar to happen to me again, so I jumped on the opportunity to own project management before I could be subjected to bad project management. Was the counterfactual of bad project management likely? Don’t know, I didn’t think about it that hard. I just wanted to not feel the pain again. All I can say is that it worked out in the end, because being a staff engineer generally requires you be good enough at project management to manage the work of at least an entire team (8-12 people).
On that point, a key experience in my path was managing an 18-month long database migration. This project was projected to cut COGS by 50% (and more than double operating margins) and improving our financials was a top company goal. I worked with 5 engineers over that time to take the project from proof of concept to roll out in production with no loss of data or downtime. This required solving a lot of problems across our stack, which made a lot of assumptions about the database layer and had insufficiently abstractions to make the process easy. Having a success like that was essential to making it to the staff level when I went for my next job. So it’s worth saying that it’s not just about project management skills, it’s about using those project management skills to generate positive outcomes for the business.
I also got really into a few other topics. One is accounting. Might sound boring, but accounting is at the heart of all businesses. You need to understand and be able to speak the language of accounting to be part of management, and it has helped me as a staff engineer. Consider this one optional, though, as I think there’s ways to be a staff engineer without much accounting knowledge (you can lean on your management partners to help you here).
I got into the idea of deliberately developmental organizations. Are DDOs a good idea? I still think probably yes, but they’re easy to get wrong. What’s important is that I spent a lot of time thinking about and then experimenting with ways to affect the culture of the organization and thereby understanding how organizations work. This is a key leadership skill you need to be a staff engineer. You have to be able to both navigate your organization and effect change in it. For example, maybe you need to foster a greater focus on quality or reliability among your engineers. This is not just about writing “better” code. It’s about shaping the org to incentivize those things through mission statements, project stack ranks, 1:1s, and winning the hearts and minds of your coworkers. You can only do this if you have some gears for how a software engineering org works.
Finally, I spent a lot of time writing. This wasn’t really for work, but clear communication is an essential skill for being a staff engineer. You’ll need to talk to a lot of people and get them to do what you want, and you can only do that if they understand what you’re asking them for. This isn’t just about writing skills, either. Fundamentally I think good writing is about having enough cognitive empathy for your audience to understand what you need to say in order to generate the desired thoughts in their minds. You’ll need to be able to do this as a staff engineer; otherwise your design docs, emails, status updates, etc. will generate more questions than they answer and that’s not acceptable to function at this level. So get good at writing and communicating.
Now, as I said, I didn’t know about staff software engineering until I was being offered jobs with that title. How did this happen? I see management and staff engineer as two sides of a coin: both are about achieving similar ends by slightly different means. Managers focus more on people and business needs. Staff engineers focus more on systems and technical constraints. Since I was functioning as a tech lead manager in my last job, it was easy to offer me a staff engineering position because it was simply specializing on one side of the management/staff divide. You don’t necessarily need to grow and then later specialize, but it’s what worked for me. Plenty of staff engineers specialize straight into it because they don’t want to be managers.
So, to sum it up, if you want to become a staff software engineer, here’s my advice:
get so good at programming you become bored
learn to operate autonomously to deliver results (become “highly agentic”)
learn how to successfully manage projects with multiple contributors
build a rich mental model of how software engineering orgs work and be able to use this to change the organization
become a skilled communicator
No.
The responsibility of Scrum Master is making sure that the rituals of Scrum are followed properly. (Makes sure there is a Retrospective. Keeps the meetings short.) Scrum Master does not care about the business case.
Project Manager is managing the business case, making project plans, drawing Gantt charts, identifying project risks—shortly, managing the project. (Scrum Master is managing the process.)
These two roles are coming from different—mutually incompatible—paradigms.
The closest analogy to Project Manager in Scrum methodology would be the Product Owner, who specifies what needs to be done and decides priorities. Scrum Master and Product Owner are two different people. (Often working for two different organizations, as Product Owner is the representative of the customer.)
In Scrum (as defined in Scrum textbooks) managers simply do not exist. And it’s not because they were rebranded to Scrum Masters. It’s because the team is organized differently, it is autonomous, and communicates directly with the customer. Which was the entire point of doing Scrum in the first place.
Many companies, sadly, keep doing the traditional project management, only using JIRA, with lower managers renamed to scrum masters, long frequent meetings renamed to agile stand-ups (where no one actually stands, because their feet would hurt), etc. And they call it Scrum, and congratulate themselves for being agile.
Okay, sure then, we were terrible at agile. Shrug? I don’t really care about what you call it. We were agile-ish, we called me the scrum master, but really my job was to make sure everything got done, was sequenced correctly, right people did the right work, etc.. I know people have some big opinions in the space of how to manage projects correctly, but my take is mostly do whatever works for your people and company to get stuff shipped. If we weren’t really agile so be it.
Using what works for you is perfectly fine. Redefining new words to mean “business as usual” makes it difficult to communicate the original idea. (In this case, the idea of developers making the decisions autonomously, without having managers. Which is what Scrum-by-the-textbook is, in a nutshell.)
Thank you for doing the work of correcting this usage; precision in language matters.
This is what I need to work on most.
I have had several important ideas which took literally years to really get acted on (proving out that they were important, incidentally) and I’m almost certain it is because I don’t have a lot of skill at overcoming the inferential distance that was (in retrospect) a huge blocker to communication, and partially, that’s because I wasn’t really thinking “okay but how is this paragraph going to land when they read it?”, instead more like “is this aesthetically pleasing to me, knowing what I know?”
Excuse me for the half-off-topic, but would you give me visibility into what might get you to work at an EA/rationality org?
Or in other phrasings:
What is preventing it?
What could change that would get you to move to such an org? Feel free to ask for magical stuff that would never happen in real life, I’m in an early “learn about the users” stage here
Thanks!
(Context TL;DR: Not enough software engineers apply to such orgs, and the orgs have no idea what’s going on since our community theoretically has so many developers. I’m trying to help figure it out)
Sure, I think it’s pretty simple in my case. I’m able to make a lot of money. I guess I shouldn’t say exactly how much, but take a look at level.fyi and look at positions equivalent to E7/L7 and that tells you something about the kind of compensation I command. Given I work at a startup (so no liquidity for my stock yet) that’s growing healthily I have a large upside with good expected value of payout. That can translate into a lot of targeted giving at funding opportunities I’m interested in prioritizing: neglected things in AI safety that are too risky/uncertain to get much other funding but that I think are interesting and deserve a shot.
To get me to apply it’d have to be clear that the work I’m doing is worth in excess of $1mm a year, otherwise I think I’m better off staying in industry and donating to fund others (since my earning potential is continuing to grow over time; if I ever top out then that might change the calculus slightly). Also, I’m going to have to be paid enough to fund the life I like living, which exists off the chunk of money I don’t donate. So in current money we’re looking at cash compensation around $300k (which also accounts for money I donate to non-EA things) and then ~$700k a year in relevant excess value generated or donation offsets to account for the difference in compensation from the alternative.
I think there’s also for me got to be some story about career trajectory. Because I’ve not topped out in my career, the role has to have a clear story about how it’s going to help me grow since I currently have the possibility of making 2-5x more than I make now in the next several years doing what I do now if I can do it a bit better. If the story is going to be that I take a few years off to work at an EA org to do good that would be fine except I’m passing up a lot of lifetime compensation in expectation, which also means passing up a lot of potential donations.
One final bit to figure in: I tried my hand a bunch of AI safety research. I produced what I think is some interesting stuff, it’s gained a little traction and others are doing things with it, but at the end of the day I like building more than anything else. So it also has to be that the job being offered is a similar software engineering job that I’ll like doing day to day. There is some value in being overqualified for a job and being able to execute on it really well, but if the org has fewer than 50-100 engineers it’s not clear there’s a lot of value for me to uniquely bring to it. Even as CTO I’m not sure there’s enough there there unless this just happens to be an EA-aligned startup with high growth potential.
Thank you very much for the high-detail answer <3
If I may continue debugging:
If an EA org would tell you that you’d produce more value for the world by working directly for them, building things, while also earning a salary (insert some number here) that lets you live well and donate to non-EA causes, even though the EA org has fewer than 50 engineers, compared to having you donate as much as you’d maximally donate ($1M/year?), would you move to the EA org?
(Reminder: I’m asking this as a “debug” question, partly to check my understanding. Pushbacks are welcome)
[Answering for myself only, though I’m in a similar position—was Principal Engineer at a FAANG for many years, recently moved to a late-stage startup in a VP-level IC role, with comp in the range that Gordon describes].
I think it’d take a lot to convince me that my comparative advantage in … the stuff described in the post (strategic tech decisions, with details down to coding and the tens of thousands of dumb little decisions that make a software product, and bridging the trust required by individual engineers and senior management) … is more valuable in an EA/Rationality org than a commercial software company.
Without some amount of market discipline and medium-term customer success indicators (denominated in dollars), I don’t think I can have the confidence and good instincts about what to change (and what to drop entirely) that make me valuable.
This is related to the discussion about how to spend EA money well—it’s not that it’s impossible to hire people, it’s that there aren’t enough well-understood projects to justify hiring expensive people. It’s not QUITE “you can’t afford me”, it’s more “you don’t have a model where you need me enough to hire me”.
(Thank you for joining in! <3)
Regarding “you don’t have a model where you need me enough to hire me”—may I ask if that’s something that an EA org (or several such orgs) have told you, or something you’re guessing?
Purely guessing. I haven’t seen any job postings or other indications that they have a large and confused enough software engineering organization to make good use of (what I think is my) main strengths.
I’ve recently changed jobs, so not looking for another change immediately, but I’d be happy to chat with execs of orgs that wonder if they need someone like me—generally that would be engineering orgs of 120-500, usually in 10-40 teams with multiple products and somewhat unaligned (heh) vision.
lol I resonate with the desire to get into a confused software engineering org and help bring order :) Though I am not an experienced manager like yourself
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100+ devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Maybe? It’d at least make it an option, but I think Dagon’s reply is right: it’d be very hard for me to evaluate if you actually need me, and if you don’t actually need me then the opportunity cost for everyone ends up being high enough that it’s probably a no.
May I ask if you evaluated this yourself? (I’m guessing: based on EA orgs being too small and with too little growth, so your own edge isn’t there, so they better hire someone else. But I’d be happy for clarity here if you’d help me out)
Yes, although not recently for any specific position. If someone came to me with a position or I saw an ad for a position (closest thing recently had been Lightcone, and I think I didn’t pursue basically for reasons above based on what I read) Is have to think about it.
I see, thanks
As I said in the other thread:
I’ll exit my user-research mode and just share that EA orgs currently have a hard time hiring devs, especially senior ones. My fairly-confident assumptions is that the devs simply don’t apply, but beyond some guesses, I wasn’t sure why. Both of you are sharing reasons which I read as mostly “the orgs are not a good fit for my skills—I’m better at managing ~100 devs, not ~3”. First of all I’d like to thank you for sharing this—it’s helping me understand this unclear picture, so, like, this is an actual thank you. :)
I expect that the EA orgs would answer your concern with “Not enough people are applying, surely not very senior people. The alternative for you working with us is that we’ll have a dev with 1-2 years of experience, who is also, eh, not ideal for the position”. And also, “you will have a higher impact working with us, including if we pay you a ton of money, compared to donating 5X what you’d donate if you’d keep working at FAANG”. I don’t know if they’d actually say this, but it is my guess (I’m currently working on figuring out this field as you understand)
May I ask for your thoughts on this?
(Maybe as a person the orgs are trying to speak to, maybe as an advisor to me who is trying to make progress on the problem)
Sorry I missed there was a question here originally, thus the late reply.
I guess my thought is put your money where your mouth is. That is, if EA orgs really want more senior folks, they’re going to have to pay for it and not rely on people being mission aligned enough to take a pay cut.
I think realistically the way to get folks is to offer more cash than they can get in the private sector (where stock is often used to offset cash compensation). So for example offer 30% more cash than 95th percentile for the level you want to hire. For example, if someone wanted to hire me to work at an EA org they’d have to pay me at least $450k/year annual cash comp by this algorithm, and thinking about it being offered that much cash given the tradeoffs of working at a smaller org where I won’t get to exercise all my skills feels fair in that I’d be even on taking that option vs. continuing at my current job.
I haven’t closely followed EA developer job postings in years, but my sense is that they fall into three categories:
WebDev
CRUD apps
ML Eng
Any of these would be a waste of a staff engineer- the first two because what the jobs require is such a small subset of what the engineer can do, and the third because it requires a bunch of skills they don’t have (while still not using the ones they do).
From a total utility perspective, it seems better that EA roles are filled by junior engineers, while senior engineers leverage their additional skills to earn more.
This doesn’t apply to absolutely every job (LW/EAF are serious works of engineering), but I do wonder if EA orgs are looking to hire more impressive people than they can actually make use of.
Startups sometimes have founders or early employees who are staff (or higher) engineers.
Sometimes this goes terribly: the staff engineer is used to working in a giant bureaucracy, so instead of writing code they organize a long series of meetings to produce a UML diagram or something, and the company fails.
Sometimes this goes amazingly: the staff engineer can fix bugs 10x faster than the competitors’ junior engineers while simultaneously having the soft skills to talk to customers, interview users, etc.
If you are in the former category, EA organizations mostly don’t need you. If you are in the latter category though, EA organizations desperately need you, for the same reasons startups want to hire you, even though you also have skills that we won’t be able to use.
If someone is considering applying to CEA I’m happy to talk with them about whether it would be a good fit for both sides, including which of their skills would be useful, which new ones they would have to learn, and how they would feel about that. Some people really like being able to meet a promising EA at a party, hear about one of their issues, rapidly bang out a PR to fix it, and then immediately see that EA become more impactful (and know that without them—it wouldn’t happen). But other people really like the “make a UML diagram” style of work, and don’t enjoy working here. It’s hard for me to guess or to give a generic answer that will fit everyone.
It seems like you are bucketing senior eng skills into bureaucracy or… agility? ability to work quickly and responsively? That is missing most of what makes staff engineers staff engineers. Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later. EA is growing but AFAIK not at nearly the speed it would take to make use of those skills: you’re more likely to end up with something terribly overbuilt for the purpose.
From what you describe, I think you’d be much better off looking for a kick-ass mid-level PM with a coding background who wants to get back into engineering. They’d be giving up much less (both in terms of money, painstakingly acquired skills they enjoy using, and future option value), and are more likely to have the skills you actually want.
> Start-ups want engineers who are overpowered for the immediate problem because they anticipate scaling, and decisions made now will affect their ability to do that later.
I’m sure this is true of some startups, but was not true of mine, nor the ones I was thinking of what I wrote that.
Senior engineers are like… Really good engineers? Not sure how to describe it in a non-tautological way. I somewhat regularly see a senior engineer solve in an afternoon a problem which a junior engineer has struggled with for weeks.
Being able to move that quickly is extremely valuable for startups.
(I agree that many staff engineers are not “really good engineers” in the way I am describing, and are therefore probably not of interest to many EA organizations.)
I’m a step lower here (senior engineer) and the reasons I and the people I know mostly don’t apply to EA orgs is that we get signals that they don’t want or value us: No remote work, generally not many actual engineer openings, various posts saying the way to get a job is to work unpaid for months in the hopes of getting attention, and at least one EA org lowballs salaries.
Thank you very much for sharing! This comment is valuable for me
What have you found in your experiments, in terms of what helps or hurts in developing DDO culture?
Lots of people are pretty suspicious of doing things that are too much like direct positive psychology work. They have their own traumas and hang ups about these sort of things and are pretty opposed to anything that looks even vaguely like group therapy. You can only do that kind of stuff if you’re the leader and you’re willing to have people quit over it (and, similarly, going forward are willing to use it as a differentiator when hiring).
What helps is just doing everyday things to create psychological safety. Listen to people. Don’t tease and taunt, even though this normally serves as a bonding activity. You can do it, but only very judiciously with people you know already feel secure and even then don’t do it in front of the people who aren’t secure. If you see other people doing things that will make others feel less psychologically safe, push back to make the workplace safer. Psychological safety is the foundation needed for development to happen.
Create space for people to try things and fail. Encourage people to take on tasks outside their comfort zone. Give them something like 70% work that they can succeed and at 30% work that they might fail at but will learn something trying. Let people have ownership and give them time to act on it. If they don’t after months, nudge them. Give them ideas. Encourage them to go out on a limb. Explicitly tell them they’re safe and you won’t hold it against them.
Are there ways to do this apart from direct experience?
After a little over a year at the startup I switched to from a Large Company, I realized that the engineering org is broken in a way that is not just “growing pains.” Levels are hazy and promotions are unclear, which makes it impossible to work towards greater responsibility / bigger problem areas. Hiring is owned by HR, so engineering has no say in this apart from conducting the actual interviews. Engineers high in the hierarchy, eg. “distinguished engineers”, are mostly focused on their own 1-person projects and almost never interact with other engineers.
I describe this because it took me over a year that all these behaviors are not merely growing pains and that I cannot do anything to change them because I am merely a “code/yaml production resource”. I am now looking for another company (in NYC!) and thinking really hard how to avoid joining another broken engineering organization.
Yes! There’s a bunch of great books out there that are helpful. The Managers Path is probably the best in some sense, but I find it’s kind of dry and easy to miss its points because of the way it’s presented. I highly recommend the blog and books of Michael Lopp. These won’t tell you everything you need to know, some of it you’ll have to figure out for yourself (even if it’s not through direct experience but thinking seriously about how to make an org that functions better), but you can learn a lot from these.
Harvard Business Review is also useful. Not because it’s going to necessarily give you a bunch of good ideas, but because it’s the equivalent of ACM Queue for managers, so you’ll get a sense of how managers think about things and that will be valuable in building up your model.
Since you consider yourself “middle management”, did anything in Zvi’s Mazes sequence resonate with your experience of the job?
I’m actually going to write a follow up post ~today addressing exactly that!
Thanks for writing this. This is exactly where I’m trying to go as my next step, but I’m not quite sure how to get there.
Is there any path to getting to this level at a smaller organization, or would I basically be required to work somewhere huge if I don’t want to top out my career at Senior Engineer?
Not really. Depends on how small though. To operate at the staff level you need approximately team scope, so you need to be setting the direction for about 8-12 engineers worth of work. If the org is small enough that this is all of engineering then there’s not a lot of room unless the CTO wants to delegate it to you so they can do something else. If you’re big enough there’s multiple teams then maybe, but there’s a lot of overlap with management and small orgs tend to ask managers to be TLMs (tech lead managers) so there’s not a lot of opportunity to demonstrate operating at the staff level.
How far into your career did you get before you decided to go down said path?
I don’t think I ever really decided, other than I’m currently choosing to be a staff engineer rather than a manager. I’m making that choice because mostly because I think at my current employer my prospects are better as a staff engineer than a manager, but I will likely flip to management at some point when the conditions make sense.