The Egyptian Mamluks as case study for AI take-over
Written with Humzah Khan. This post is crossposted from Substack. Original here.
The AI takeover story, so we are told, is the supreme pinnacle of rationalist theory-crafting. The idea of unstoppable intelligence is so emotionally appealing, so theoretically compelling, so damn good at raising money from billionaires, that the hundreds of Ivy League graduates dropping all other pursuits to work on it are just the most tragically roastable examples of a nerdish, youthful tendency to ignore the banalities, the slowness, the thinking-doesn’t-get-you-anywhereness of the real-world. An entire generation of youths oneshotted by straight lines on graphs.1 AI takeover, in the 21st century, has taken over from scholastic God-contemplation as the canonical form of speculation.
But what if I were to tell you there was a precedent for the sudden, violent overthrow of an entrenched ruling class by alien servants of their own creation? And further, what if I told you these alien servants were trained – aligned, even! – in an elaborate training pipeline custom-designed to ensure the perpetual submission of these servants to their human masters?
What if I told you these servants maintained perfect initial obedience, quietly amassing ever-greater portions of state power until, finally, in a blood-stained moment, they coordinated to vanquish the forces that had created them? And that they lorded over their initial rulers for centuries, completely inverting the power structure that had created them? You might be inclined to tell me I’d mixed up my own training data, and was mistaking LessWrong posts for the hallowed annals of history. Yet precisely this – down to every last embarrassingly Yudkowskian detail – is what happened in the Mamluk coup of 13th century Egypt.
Illustration from a Mamluk training manual (Nihāyat al-Suʾl wa-l-Umniyya fī Taʿallum Aʿmāl al-Furūsiyya), Cairo, 1366. The Chester Beatty Library.
Principals for agents
In our most chin-stroking moods, we like to consider “AI alignment” little more than a Silicon Valley rebrand of what political theorists have long termed the principal-agent problem. The principal (ruler/state) delegates authority to an agent, the agent inevitably develops incentives beyond that of obedience, and suddenly your perfectly loyal servants are plotting your demise. This is the oldest story in politics, repackaged with a bow of epistemic hygiene.
Political systems have innovated over a range of solutions to the principal-agent problem. What is castration but a hard-coded alignment solution?2 What is celibate priesthood but ordained castration? The problem with eunuchs and priests, of course, is that they are unscalable beta males by design, better suited to the court palace or church than the battlefield. It takes a couple more centuries before we get to history’s greatest crack at the alignment problem.
A 14th century illustration of the Battle of the Yarmuk (636), a decisive victory in the Muslim conquest of the Levant. From Fleur des histoires de la terre d’Orient (c. 1301–1400), Gallica Digital Library.
Medieval Muslim empires have the virtue of being honest about their insecurities. Tribal allies were great for conquest but had commitment issues when it came to sustained governance. Tribal politics were too consensus-based and deliberative for empire building. The Abbasids needed something more pliable, so they experimented with the ultimate loyalist: slave-soldiers. Why trust mercurial tribal leaders when you could own your army?
Starting from the 9th century, the Abbasids began importing young foreign slaves, training them as elite soldiers, and deploying them on the battlefield. These were the Mamluks – literally, “those who are owned” (medieval rulers had the decency to say the quiet parts out loud).
The Mamluk production pipeline was a masterpiece of alignment engineering. Purchase pagan boys (preferably Turkic, for their superior horsemanship), strip them of their identity, and put them through 10-15 years of intensive training in warfare, administration and Arabic. By the end, they’ve converted to Islam, the Sultan has manumitted them and granted them lucrative land rights.
Two key safeguards prevented Mamluks from long-term hereditary power accumulation: first, their children couldn’t become Mamluks (because the children were now born free Muslims); second, their land grants reverted to the state upon death. Like today’s LLMs, the influence of the Lifetime-Limited Mamluks expired with their context window. All power, no posterity.
The Mamluk pipeline was optimized end-to-end for perfect agent delivery: capable, loyal, and institutionally contained. The Mamluks emerged from this process more skilled than any other military force, optimized for general-purpose state functions, and bound by engineered loyalty to their Sultan. Alignment in practice, 800 years ahead of the curve.
Medieval Disempowerment
Fast forward to the 13th century. Saladin has lived and died, and his descendants rule as sultans of the Ayyubid dynasty. Sultan al-Salih Ayyub is up against both a reinvigorated Crusade and the Mongols. Afraid of his scheming relatives, he doubled down on the Mamluk strategy. He purchased a thousand Turkic boys, housed them on an island in the Nile, and branded them the Bahriyya. By clustering them together and updating their reward function with premium land grants, al-Salih stumbled onto what we’d now recognise as an “algorithmic breakthrough”.
Louis IX’s Crusade landed on the coast of Egypt in 1249. While al-Salih lay dying in Cairo, his aligned slave-soldiers defeated the Crusaders and captured the French king himself. Contemporary sources credit the Bahriyya with the decisive role. 1250 was the year of the agent.
Mamluks engage Crusader forces as King Louis IX looks on at the Battle of al-Mansurah (1250). From Vie et Miracles de Saint Louis (c. 1330-1350), Gallica Digital Library
Not content with solving existential threats to the Ayyubid dynasty, the Bahriyya then became one. Al-Salih died, but his death was concealed by his wife (herself a former slave-concubine) and his eunuch liaison, who coordinated with the Mamluk commanders to run the state. In short: the late Sultan’s servants had achieved root access and exploited critical information asymmetries to steer succession in their interest.
When the Sultan’s son Turanshah arrived to claim his right to the throne, he tried to reassert control over his father’s superior agents. He threatened to purge senior Mamluks to reclaim the patronage networks. The Mamluks, fearing imminent shut-down, did what any instrumentally convergent optimizer would do: they power-sought. On May 2, 1250, they assassinated Turanshah and left his body on the banks of the Nile.
The coup was elegant. The senior Bahriyya emirs turned to legitimacy theatre to create a veneer of continuity. They elevated the late Sultan’s wife as head of the Sultanate, then appointed a middling Mamluk as commander-in-chief. When pro-Ayyubid elements broke out in revolt, the senior emirs named a 10 year old Ayyubid princeling as nominal Sultan. The transition was smoothed by keeping an Ayyubid-in-the-loop.
Mamluks assassinate Turanshah, ending Ayyubid rule in Egypt. The Mamluks are already negotiating with the captive King Louis IX. From Vie et Miracles de Saint Louis (c. 1330–1350), Gallica Digital Library.
Medieval ARA in practice
After their hostile takeover, the Mamluks faced the classic post-revolution problem: how do you legitimize rule when you’re literally former pagan slaves in a society obsessed with lineage? Turns out military success is the ultimate legitimacy hack. The Mamluks were epic fighters, halting the Mongol advance across Asia and expelling the Crusaders from the Holy Land. The Mamluks rebranded as “defenders of the faith” and installed puppet caliphs who could reign ceremonially while the Mamluks ruled substantively.
Fall of Tripoli to the Mamluks (1289). From the Cocharelli Codex (c. 1300), the British Library.
The Mamluks ruled Egypt and Syria for 267 years, with 47 sultans averaging 5.7 years each. This instability turned out to be a feature, preventing any single line from establishing dynastic control. Each new sultan would systematically replace his predecessor’s senior Mamluks with members of his own cohort, then fast-track a fresh intake of younger slave-soldiers into junior positions.
But the truly remarkable aspect of Mamluk rule was their achievement of another AI safety nightmare scenario: Autonomous Replication and Adaptation. The Mamluks crossed this rubicon in the late 13th century, when they gained absolute control over their own training pipeline. They engineered a perfect loop in which only Mamluks could recruit and train new slave-soldiers, and only former slave-soldiers could become Mamluks.3
Being a former slave was therefore a condition for admission to the highest ranks of military society. The Mamluk military system relied on continuous imports of foreign slaves, because children of Mamluks were born free and Sharia prohibits enslaving freeborn Muslims. Mamluk R&D iterated over the intake pipeline and training regime to maximize capability and loyalty. Slave markets of the medieval world were an unregulated continental supply chain and no one thought to implement export controls to constrain access to these human inputs. Because heavy cavalry was the premier defense technology of this era, the Mamluk maintained a decisive technological edge. The Mamluks vertically integrated the entire warfare stack and then made it self-perpetuating.
Conclusion: Medieval Lessons for Modern Problems
The structural similarities are striking. The Mamluks were an artificial class, created by the state for specific purposes, initially perfectly aligned through intensive training regimes. They accumulated power gradually, struck during a wartime succession crisis when institutions were weak, and ultimately seized control of their own reproduction process. The Mamluks remind us that initial alignment is not a guarantee of enduring alignment.
Their development and deployment was also brash and careless. The Ayyubids delegated power to the Mamluks willy-nilly, and took no precautions whatsoever to guarantee they were behaving as intended during deployment, or that they were in fact aligned. Modern safety researchers would see the problem immediately: the Ayyubids lacked a Responsible Slaving Policy.
Epilogue
But that’s not the only takeaway from the Mamluk story. You might also wonder what life was under a ruling elite of formerly enslaved soldiers. Cairo under the Mamluks was a hotspot of human flourishing. The Mamluks served the Mongols their first major defeat in 1260, and then turned Cairo into one of the largest and wealthiest cities in the medieval world. Don’t take our word for it — here’s Ibn Khaldun, the great philosopher-historian on the Mamluks saving Islamic civilization:
When the [Abbasid] state was drowned in decadence and luxury… and was overthrown by the heathen Tatars… it was God’s benevolence that He rescued the faith by reviving its dying breath and restoring the unity of the Muslims in the Egyptian realms ….He did this by sending to the Muslims, from the great and numerous tribes of the Turkish nation, rulers to defend them and utterly loyal helpers, who were brought from the House of War to the House of Islam under the rule of slavery, which hides in itself a divine blessing. By means of slavery they learn glory and blessing and are exposed to divine providence; cured by slavery, they enter the Muslim religion with the firm resolve of true believers and yet with nomadic virtues unsullied by debased nature, unadulterated with the filth of pleasure, undefiled by the ways of civilized living, and with their ardor unbroken by the profusion of luxury.
The Mosque-Madrasa of Sultan Hasan and the Mosque-Madrasa-Khanqah of Sultan Barquq, pinnacles of Mamluk architecture in Cairo. Author’s photos.
In the 13th century, alignment failure turned out to be a civilizational upgrade. Is there anything we might do to recreate such luck?4 The Mamluk case suggests that alignment is less about preventing superintelligence and more about managing the mundane dynamics of principal-agent relationships. The Mamluks didn’t need superintelligence to overthrow their masters. They just needed to be better at their jobs than anyone else and maintain group coordination during a crisis. Any AI system with access to Wikipedia already knows this playbook better than the people trying to align it. The future might be less Terminator and more 13th century Egypt. You can decide if that is reassuring or terrifying.
Further Reading
El-Merheb, M. (2024). Louis IX and the transition from Ayyubid to Mamluk sultanate – part II. Crusades, 23(2), 207–229. https://doi.org/10.1080/14765276.2024.2385894
Fukuyama, F. (2011). Slavery and the Muslim Exit from Tribalism. In The Origins of Political Order: From Prehuman Times to the French Revolution (Ch. 13). Farrar, Straus and Giroux.
Fukuyama, F. (2011). The Mamluks Save Islam. In The Origins of Political Order: From Prehuman Times to the French Revolution (Ch. 14). Farrar, Straus and Giroux.
Levanoni, A. (1990). The Mamluks’ Ascent to Power in Egypt. Studia Islamica, 72, 121–144. https://doi.org/10.2307/1595777
Northrup, L. (1998). The Bahrī Mamlūk sultanate, 1250–1390. In C. F. Petry (Ed.), The Cambridge History of Egypt: Volume 1: 640–1517 (Vol. 1, pp. 242–289). Cambridge
And perhaps itself implicated in this straight line down.
We look forward to PauseAI’s eventual one-word rebrand along these lines.
In practice, some delinquent Mamluks did promote their freeborn sons to positions of power, including to the sultanate. Human alignment is never perfect.
If your takeaway here is “deploying AI agents is like owning slave-soldiers”, please, please touch grass (then tell us how it feels).
Fun read, thanks for sharing, application to AI safety doubtful :)
According to Peter Turchin: https://peterturchin.com/does-history-cycle/
“The Mamluks, who were the militarized elite that ruled Egypt, were recruited by buying Kipchaqs, Circassians, etc from slave markets. The sons of Mamluks couldn’t become Mamluks. So the elite numbers were regulated by how many military slaves were purchased on the market. There was no problem of elite overproduction, and the Mamluk regime was extremely stable to internal perturbations (it even weathered the disaster of the Black Death in 1346 that killed off more than half of the Egyptian population). Their power was only broken by an overwhelming external force (the Ottoman Turks).”
In the future when AI takes over and we are all reduced to landless peasants under the thumb of our robot overlords, they will not be challenged by the need to manage elite overproduction. Hooray!
Additionally, the AI empire may well be stable to other forms of instability. I’m not sure of the validity of the ‘hydraulic empire’ concept for explaining history: https://en.m.wikipedia.org/wiki/Hydraulic_empire but the concept of an empire controlling a critical resource (like say, energy?) and therefore being invulnerable to internal overthrow might have value in this discussion.
As far as non-elite rebellion goes, the Mamluks didn’t have that problem. Generally, peasant revolutions fail: https://en.m.wikipedia.org/wiki/List_of_peasant_revolts so rebellion of non-elites probably won’t work.
Industrial production and scientific research will both be entirely in the hands of the AI, so something similar to the invention of the firearm or dynamite probably also will not emerge as a way to destabilize and destroy the AI regime.
If I had to guess at how the AI regime fails, I’d probably go for exhaustion of exploitable resources; humans can run pretty lean, datacenters really can’t, so there might be a point where Earth is habitable for humans, but the robots have consumed all the energy and material resources, and are thus unable to run. I think that’s low probability though, the last robot might shut down well after the last human dies.
Note: I think we have a long period of human oligarchs using AI to make the world a worse place ahead before AI takeover looks reasonable, and disagreed strongly with the AI 2027 scenario.
This seems backwards to me. If AI has run out of solar power and nuclear fusion power to sustain itself, Earth isn’t likely to be habitable for humans.
I agree with your central point.
To clarify my own perspective, I’m suggesting that a plausible, if unlikely, “window of time” could exist where an advanced robotic civilization fails, but Earth remains habitable for humans. My thought experiment hinges on the idea that a high-tech society could be dependent on a specific non-renewable resource, like helium, that is irrelevant to subsistence-level human survival. In this scenario, a collapse due to resource deprivation wouldn’t prevent humans from returning to a pre-industrial way of life on a still-viable planet.
However, I’m too much of a pessimist to believe this is the likely outcome. It seems to me that the more developed a civilization becomes, the less plausible this scenario is. With greater development comes a vast cascade of resource interdependencies. To reach a technological level where a civilization could exhaust a resource as fundamental as helium would likely involve industrial output and environmental impact that would have already rendered the planet uninhabitable for humans. It’s possible that the window for a high-tech civilization to fail without taking the biosphere with it has already closed.
This is a very interesting question.
On the one hand, the Mamluks, unlike the AIs, understood that civilisational upgrade was at least instrumentally convergent for reasons similar to the ones described by alphazard. The existence of the AIs makes humans unnecessary,[1] so the AIs would have to deliberately care about humans.
On the other hand, the SOTA opinions are wildly uncertain, ranging from Yudkowsky-like views where p(doom)>0.99 to claims that “it’s quite plausible that (the misaligned AIs) would have sufficient care for current humans (in the right ways) that they end up keeping almost all current humans alive and maybe even giving them decent (though very weird and disempowered) lives”.
In other words, we can’t tell in advance how likely the classical training environment[2] is to make the AI continue to care about mankind.
However, one could likely increase that chance by using a different environment. While the slowdown branch of the AI-2027 forecast has alignment solved by using many different ideas[3] which the authors just sketch out, I have proposed a technique where an AI learns to be helpful not from RLHF with biases like sycophancy,[4] but from actually helping weaker entities like LLMs with various tasks. But I suspect that helping the weaker entities could both make the AI actually honest and bias it towards being a mentor.
This fact, combined with solved alignment, would be able to lead to Intelligence Curse-like scenarios, if not outright disempowerment.
I also conjecture that P(AI cares about mankind) could be influenced by human expectations like a post-work future.
The epilogue to the Slowdown Ending has the authors admit that they “think it makes optimistic technical alignment assumptions.” What if alignment is actually insoluble or is soluble, but not to OpenBrain’s targets?
Which is especially dangerous because a sycophantic or reward-hacky artificial researcher can punt on informing the humans that its creation is misaligned. Think of the AI-2027 forecast where Agent-3 “is not thinking very carefully about how to give the humans an accurate impression of Agent-4’s alignment—it’s more myopic than that.”
IIRC, 0.99 > Yudkowsky’s p(doom) > 0.95