Recursive Middle Manager Hell: AI Edition
Two-sentence summary
In Recursive Middle Manager Hell, Raemon argues that organisations get increasingly warped (immoral/maze-like/misaligned/...) as they gain more levels of hierarchy.[1] I conjecture that analogous dynamics also arise in the context of AI automation.
To be frank, I think 95% of the value of my post comes just from the two-sentence tl;dr above. If you haven’t done so yet, I recommend reading Recursive Middle Manager Hell and then asking “how does this change in the context of automating jobs by AI”. But let me give some thoughts anyway.
Automation so far
[low confidence, informal] It seems to me that automation so far mostly worked like this:
Mostly on the level of individual tasks (eg, “Google creating a calendar event based on me receiving flight tickets via email”, as opposed to “Google automatically handling all of my calendar events”).
Mostly without very long chains with no human in the loop (eg, the calendar example above, but mostly not long chains of Zapier macros).
Mostly trying to imitate humans, rather than find novel solutions (eg, when Google puts a flight into my calendar, it does the same steps I would do, except automated. When Excel computes stuff for me, it likewise just performs calculations that I came up with.).
I think this is starting to change. On the abstract level, I would say that (3) has changed with the introduction of ML, and (1) & (2) will now be changing because LLMs are growing more general and capable.
Examples
But perhaps it is better to instead give more specific examples of “recursive middle automaton hell”:
We already have some companies using ML to do the initial screening of CVs for job applications. And some companies might also use AI to do the same for (job-application) cover letters etc. Similarly, some people are starting to use LLMs for fine-tuning their CVs or writing their cover letters. But this is still somewhat rare, so most of these interactions are still human-human or human-AI. What happens once most of these interactions are with black-box AIs on both sides?
One of the joys of academic work is [writing recommendation letters, reading recommendation letters, writing grant proposals, reviewing grant proposals, writing grant reports, and reviewing grant reports. Aaaah!]. Most work in each of these tasks is boring, annoying, and monotonous. Let’s automate it, hurray! … but what happens once both sides of this are automated?
Both of the examples above only automate two levels of “hierarchy”. And I have never worked for a large corporation, so I don’t have good intuitions for how things are likely to go there. But I expect we will see longer chains, of automating poorly specified tasks, using black-box models, with unclear feedback on the overall process. What happens then?
What next?
In a sane world with lots of time to investigate this, this sounds like an exciting research topic for game theory, mechanism design, and whole lot of other fields! In the actual world, I am not sure what the implications are. Some questions that seem important are:
Which assumptions are needed for “recursive middle manager hell”, and which of those are likely to hold for AI?
In particular, how likely are we to see long chains of automation, with no human in the loop?
If “recursive middle automaton hell” is a thing, how is it going to interact with the human version, “recursive middle manager hell”? (Will it displace it? Make better? Will the negative effects add up, additively? Will the effects feed on each other, making things go completely crazy?)
- ^
The proposed mechanism for this is that the only pieces of the organisation that are “in touch with reality” are the very top and bottom of the hierarchy (eg, the CEO/the people creating the product), while everybody else is optimising for [looking good in front of their direct supervisor]. Because of various feedback-cycles, this—the claim goes—results in misalignment that is worse than just “linear in the depth of the hierarchy”.
For a better description of this, see Raemon’s Recursive Middle Manager Hell or Zvi’s (longer and more general) Immoral Mazes.
As a manager (and sometimes middle manager) I’ve been thinking about how LLMs are going to change management. Not exactly your topic but close enough. Here’s my raw thoughts so far:
the main reasons you need management layers is coordination
one person can’t directly manage more than about 12 people effectively
past that point you need someone to hide some details for you
with ai though one director can manage more managers
normally coordination costs make it hard to manage more than 6 managers
llms may allow a director to manage 12 managers/teams by automating comms tasks that normally require process (meetings)
less meetings mean you can coordinate more work at once
this lets the org stay flatter longer, which reduces the amount of coordination hell
many manager tasks are house keeping
managers spend 80% of their time on house keeping so they have the info and org they need to do the critical work 20% of the time
some of the house keeping can be automated with llms
for example, no need to hear all the status reports if an llm can summarize them and pick out the bits you need to act on and hide the parts you can safely ignore, but still know about them and tell you about them if they become relevant
the human touch matters
humans respond to other humans
as a manager much of what makes you successful is your ability to lead, i.e. to get other humans to trust you enough to do what you tell them to do
llms are unlikely to be leaders in the short term unless they get way better real fast
llms are more like assistants; they lack executive function
understanding what llms can do for your people
you’re going to need to understand how and when your ICs should use llms to get their job done
you’ll have to identify when people are inefficient becuse they don’t use ai well and then help them learn to use it
this is a totally new thing no manager has done before, but we’re going to have to figure it out
we’ll probably get llms to help us do it
Very interesting points, if I was still in middle management these things would be keeping me up at night!
One point I query is “this is a totally new thing no manager has done before, but we’re going to have to figure it out”—is it that different from the various types of tool introduction & distribution / training / coaching that managers already do? I’ve spent a good amount of my career coaching my teams on how to be more productive using tools, running team show-and-tells from productive team members on why they’re productive, sending team members on paid training courses, designing rules around use of internal tools like Slack/Git/issue trackers/intranets etc… and it doesn’t seem that different to figuring out how to deploy LLM tools to a team. But I’m rusty as a manager, and I don’t know what future LLM-style tools will look like, so I could be thinking about this incorrectly. Certainly if I had a software team right now, I’d be encouraging them to use existing tools like LLM code completion, automated test writing, proof-reading etc., and encouraging early adopters to share their successes & failures with such tools.
Does “no manager has done before” refer to specific LLM tools, and is there something fundamentally different about them compared to past new technologies/languages/IDEs etc?
I think the general vibe of “this hasn’t been done before” might have been referring to fully automating the manager job, which possibly comes with very different scaling of human- vs AI managers. (You possibly remove the time bottleneck, allowing unlimited number of meetings. So if you didn’t need to coordinate the low-level workers, you could have a single manager for infinite workers. Ofc, in practice, you do need to coordinate somewhat, so there will be other bottlenecks. But still, removing a bottleneck could changes things dramatically.)
My first guess is that the sort of problems I expect from here are sufficiently weirder/differenter from the classic human middle-manager-culture problems that I think it’s probably better to just Original See on the topic rather than try to check whether the Recursive Middle Manager Hell hypothesis in particular holds here.
(I do think it’s an interesting/important question though)
This sounds really interesting! Generally it seems that most people either believe AI will get power by directly being ordered to organize the entire world, or it will be some kind of paper-clip factory robot going rogue and hacking other computers. I am starting to think it will more likely be: Companies switch to AI middle managers to save $$$, then things just happen from there.
Now one way this could go really mazy is like this: All AI-models, even unique ones custom made for a particular company are based on some earlier model. Let’s say Wallmart buys a model that is based on a general Middle Manager Model.
This model now has the power to hire and fire low-level workers, so it will be very much in their interest to find out what makes the model tick. They can’t analyse the exact model (which is being run from a well guarded server park). But at some point somebody online will get hold of a general Middle Manager Model and let people play with it. Perhaps the open-source people will do all sorts of funny experiments with it and find bugs that could have been inherited by the Wallmart model.
Now the workers at Wallmart all start playing with the online model in their spare time, looking around AI-forums for possible exploits. Nobody knows if these also work on the real model, but people will still share them, hoping to be able to hack the system: “Hey, if you try to report sick on days when you anyway had time off, the Manager will give you extra credits!” “Listen, if you scan these canned tomatoes fifty times it triggers bug in the system so you will get a higher raise!” Etc.
The workers have no way to know which of the exploits work, but everybody will be too afraid of loosing their jobs if they are the only one NOT hacking the AI. Wait for a few years and you will see the dark tech-cult turning up.
Evolution gives us many organically designed Systems which offer potential Solutions:
white blood cells move everywhere to spot and kill defective cells with literal kill-switch (apoptosis)
A team of Leucocytes (white blood cells):
organically checking the whole organisation at all levels
scanning for signs of amorality/misalignment or any other error
flagging for surveillance, giving warnings, or sending to a restorative justice process depending on gravity
agents themselves held to the highest standard
This is a system that can be implemented in a Company and fix most of the recursive middle manager hell.
Many humans would not like that (real accountability is hard, especially for middle-men who benefit from status quo), but AI won’t mind.
So the AI “head” could send their Leucocytes check all the system routinely.
I agree that the general point (biology needs to address similar issues, so we can use it for inspiration) is interesting. (Seems related to https://www.pibbss.ai/ .)
That said, I am somewhat doubtful about the implied conclusion (that this is likely to help with AI, because it won’t mind): (1) there are already many workspace practices that people don’t like, so “people liking things” doesn’t seem like a relevant parameter of design, (2) (this is totally vague, handwavy, and possibly untrue, but:) biological processes might also not “like” being surveiled, replaced, etc, so the argument proves too much.
(1) « people liking thing does not seem like a relevant parameter of design ».
This is quite a bold statement. I personally believe the mainstream theory according to which it’s easier to have designs adopted when they are liked by the adopters.
(2) Nice objection, and the observation of complex life forms gives a potential answer :
All healthy multicellular cells obey Apoptosis.
Apoptosis literally is « suicide in a way that’s easy to recycle because the organism asks you » (the source of the request can be internal via mitochondria, or external, generally leucocytes).
Given that all your cells welcome even literal kill-switch, and replacement, I firmly believe that they don’t mind surveillance either!
In complex multicellular life, the Cells that refuse surveillance, replacement, or Apoptosis, are the Cancerous Cells, and they don’t seem able to create any complex life form (Only parasitic life forms, feeding off of their host, and sometimes spreading and infecting others, like HeLa).
Fair point. I guess “not relevant” is a too strong phrasing. And it would have been more accurate to say something like “people liking things might be neither sufficient nor necessary to get designs adopted, and it is not clear (definitely at least to me) how much it matters compared to other aspects”.
Re (2): Interesting. I would be curious to know to what extent this is just a surface-level-only metaphor, or unjustified antrophomorphisation of cells, vs actually having implications for AI design. (But I don’t understand biology at all, so I don’t really have a clue :( .)
(1) « Liking », or « desire » can be defined as « All other things equal, Agents will go to what they Desire/Like most, whenever given a choice ». Individual desire/liking/tastes vary.
(2) In Evolutionary Game Theory, in a Game where a Mitochondria-like Agent offers you choice between :
(Join eukaryotes) mutualistic endosymbiosis, at the cost of obeying apoptosis, or being flagged as Cancerous enemy
(Non eukaryotes) refusal of this offer, at the cost of being treated by the Eukariotes as a threat, or a lesser symbiote.
then that Agent is likely to win. To a rational agent, it’s a winning wager. My last publication expands on this.
Some thoughts (don’t have any background in stuff related but seemed interesting).
I think it would be interesting to see what you found if you looked into the state of existing research on AI coordination / delegation / systemic interactions and if any of it feels related. I’d be mildly surprised if people have studied exactly this but expect many relevant posts/papers.
In terms of related stuff on LessWrong, I can’t find it now but Paul Christiano has a post on worlds where things go badly slowly and I think this would be kinda in that genre.
I think this is an interesting thing to consider and feels somewhat related to Dan Hendrycks Natural “Selection Favors AIs over Humans” https://arxiv.org/abs/2303.16200. The connection in my head is “what does an AI ecosystem look like”, “what does it mean to discuss alignment in this context”, “what outcomes will this system tend towards” etc. The same way middle managers get selected for, so more generally AI systems with certain properties get selected for.
You might want to read about Ought’s agenda with supervise processes not outcomes which feels relevant.
Recursive middle manager hell feels somewhat related to inner misalignment / misaligned mesa-optimizers where instead of being a subset of the processing of an LLM (how I normally think about it but maybe not how others do), you have your AI system made of many layers and it’s plausible that intermediate layers end up optimizing proxies for inputs to what you care about and not even the thing itself. In this view, it seems like the misalignment of middle managers which usually makes companies less effective might just lead to selection against such systems as compared to systems with less of these properties.
There might be some strategically valuable research to be done here but it’s not super clear to me what the theory of change would be. Maybe there something to do with bandwidth / scalability tradeoffs that affect how tightly coupled vs diffuse/distributed useful/popular AI systems will be in the future.