Here, at the very beginning of our story, I need to highlight two seemingly boring and mundane points. These points are:
1. Large organizations are immensely powerful in modern society
2. They exhibit their own emergent behavior
These are my starting claims. I assume these as obvious points upon brief reflection. One so obvious as to be commonplace in our language. For example, these claims apply to the large organizations that constitute collective “Big Government”, “Big Business”, and “Big Science.” We speak and write as if this is a generally accepted fact about our world. We say things like “The US did this for that reason,” “Facebook (Meta?) did that to maximize its profit,” “The National Science Foundation decided to fund research for the benefits it perceived over and above the potential costs.” These two claims, I think, all on their own, should cause us pause for concern when we consider the sheer size and reach of our modern organizations (something we’ll return to in a future post). Given their size, complexity, and emergent behavior, these increasingly powerful organizations are also demonstrably unaligned with the general interests of human freedom, human intelligence, and human love.
These two claims need to be taken together with one additional claim, a third important point for our story:
3. These large organizations systematically deploy machine agents to accomplish their goals
These three claims, I argue, point to the current existence and power of superintelligent hybrids. Superintelligences that are already beyond human control. These superintelligences are the large hybrid organizations of people and machines. These hybrid entities are currently more intelligent, they have more memory and more compute among other capabilities and capacities, than any human or human organization had before the incorporation of machine agents.
It is these superintelligent hybrids, with their coupling of new forms of increasingly intelligent and capable machine behavior that may have the most dramatic impact in directing the course of humanity’s future. This is a troubling conclusion, one that I believe should be a major concern of those concerned with human dignity and particularly those actively influencing AI policy, research, development, and deployment. These concerns are playing out now, not at some hypothetical point in the future. These superintelligences are shaping and manipulating our preferences, desires, and attention, in mass, now. And, they already exceed our ability, as individual humans to fully control them.
It is with all of this in mind, that I argue that we, the humanity we, have lost control over the hybrid superintelligences amongst us. We have not, it does not seem, lost total control. But, we have lost a frighteningly larger share of control than collectively we have internalized.
I will build this argument out further in a future post by visiting with Aldous Huxley’s 1958 observations, in A Brave New World Revisited, in particular with Huxley’s observations that the world suffers from trends towards over-population and over-organization, and that these two trends have led to great advances in authoritarianism and mass manipulation by large organizations in the forms of Big Government and Big Business (and sometimes even Big Science!). Following the description of these challenges, I will argue that another trend, that of over-digitalization, has begun to amplify and exacerbate authoritarianism and mass manipulation. These three challenges of over-population, over-organization, and over-digitalization have already greatly altered the relationships between humans, their minds, their environments, their machines, and their organizations.
However, before turning to Huxley’s observations in further detail, we will first visit with an abridged version of Olaf Stapledon’s The Other Earth and see the great tragedy of the Other Men. It is here that we see a picture of a world so different, yet so strikingly similar, to our own. The story is one of eternal radio bliss, or the pleasure of a tailored “news feed.” At the end of Part I of this sequence, you will meet Analogia. Analogia makes the point well enough, that in our time, the “radio” itself begins to take control.
These initial explorations will set us down our journey to understand what is at stake in our current, mad world. The facts on the ground suggest that superintelligence is here, that it is not perfectly aligned with the interests of humanity, and that we have already lost significant control of it.
The ultimate consequence or dominating end of over-population, over-organization, and over-digitalization may therefore be, I will argue, the death of freedom and with it the great beauties of love, humanity, and even the very utility of intelligence.
As we delve into these global trends, it will be necessary to dive a bit further into the nature of humans, machines, and hybrids. To do this we’ll need to discuss organizations themselves as structures inhabited by complex agents and to compare organizations to organisms in the ways that they are generally constituted. Keeping the focus on organizations and institutions we’ll turn to Max Weber, Herbert Simon, Elinor Ostrom, and others for a picture of human organizations and how they navigate and manipulate the complexities of their environments, distributing authority and consuming resources throughout.
Even by Aldous Huxley’s time we lived in a world dominated by the delegation of much of our private authority to increasingly powerful and large organizations. The most obvious examples still being modern nation states and large multinational corporations. And, I will argue that for most, if not all of us, these organizations now exert even more authority and control over our lives than they did in Huxley’s time.
Over-population is a topic that seems to have been absorbed out of the modern dialogue on global problems. However, I believe it is a mistake not to put this fact out front as a leading cause of over-organization and over-digitalization. In the post on over-population, I will argue that while the global population growth rate itself has slowed, the sheer number of new humans has been a leading cause in straining the capacities of the planet, our nations, and our organizations, all of which has left us with both an increasing need for more organization (leading to over-organization) and (often) increases in suffering throughout society.
From our discussion of over-population and over-organization, we will turn to over-digitalization and the failure of cybernetics, that is, just with over-population and over-organization, we have lost significant control of the process of digitalizing our analog world. The earliest AI pioneers Alan Turing, John von Neumann, Norbert Wiener, and others worried over the questions of governing or retaining control of machines and machine behavior embedded in human society. Their concerns were justified, as we now live in Our Brave New World in which over-digitalization has crashed into over-organization and over-population to create a truly frightening path to Analogia. A world George Dyson describes thus:
“In the fourth epoch of technology, the powers of the continuum will be claimed by machines. The next revolution, as fundamental as when analog components were assembled into digital computers, will be the rise of analog systems over which the dominion of digital programming comes to an end. Nature’s answer to those who seek to control nature through programmable machines is to allow us to build systems whose nature is beyond programmable control.
Nothing is to be gained by resisting the advance of the discrete-state machines, for the ghosts of the continuum will soon return, when the grass is eight inches high in the spring. A world shared with true machine intelligence, proliferating in the wild as opposed to the captive, domesticated versions being marketed today, will bring the new millennium that Wovoka and others predicted, infused by an intelligence that belongs not to us but to Nature as a whole.
The human species, and the human mind, grew up in a world shared with creatures large and small, animated by spirits that some were privileged to communicate with but no one claimed to understand, let alone control. ‘The least particle must be considered as a world full of an infinity of different creatures,’ Leibniz admitted in 1693.
Some see only an empty hat. Others see the full power of the continuum, with the powers of the merely countable infinities becoming small fishes in the rivers while megafauna once again roam the earth.
Not even ashes will remain.”
— Analogia: The Emergence of Technology Beyond Programmable Control by George Dyson
Dyson’s argument, which we will dive further into in a later post, I find compelling.
It seems to me that we find ourselves currently living with the introduction of machine agents into our over-populated, over-organized, and over-digitalized world. We are not at the precipice, we have leapt headfirst over the unknown horizon.
The question that immediately presents itself is: what do we do about this?
Here we will find no simple answers. We will see a complex world that is inhabited by machine intelligence, superintelligences, and digital entities beyond programmable control, that, hopefully, we can use to point us away from worse and world-ending directions and towards directions that encourage human freedom, love, and intelligence.
Now that I have painted some of the general contours for you, here and in the sequence introduction, let us spend a few more moments on the consequences of hybrid superintelligences amongst us mere mortals.
Norbert Wiener, early in the rise of AI, paints well the concern for humans of over-organization and the rise of superintelligent hybrid organizations that dominate humans:
“I have spoken of machines, but not only of machines having brains of brass and thews of iron. When human atoms are knit into an organization in which they are used, not in their full right as responsible human beings, but as cogs and levers and rods, it matters little that their raw material is flesh and blood. What is used as an element in a machine, is in fact an element in the machine. Whether we entrust our decisions to machines of metal, or to those machines of flesh and blood which are bureaus and vast laboratories and armies and corporations, we shall never receive the right answers to our questions unless we ask the right questions. . . . The hour is very late, and the choice of good and evil knocks at our door.—NORBERT WIENER, The Human Use of Human Beings”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
It is with this inspiration from Wiener, that Daniel Hillis, offers some insight into our current situation, one surrounded and dominated by hybrid superintelligences:
“What is now clear, whether or not it was apparent to Wiener, is that these organizational superintelligences are not just made of humans, they are hybrids of humans and the information technologies that allow them to coordinate. Even in Wiener’s time, the “bureaus and vast laboratories and armies and corporations” could not operate without telephones, telegraphs, radios, and tabulating machines. Today they could not operate without networks of computers, databases, and decision support systems. These hybrid intelligences are technologically augmented networks of humans. These artificial intelligences have superhuman powers. They can know more than individual humans; they can sense more; they can make more complicated analyses and more complex plans. They can have vastly more resources and power than any single individual.”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
Hillis continues:
“Although we do not always perceive it, hybrid superintelligences such as nation-states and corporations have their own emergent goals. Although they are built by and for humans, they often act like independent intelligent entities, and their actions are not always aligned with the interests of the people who created them. The state is not always for the citizen, nor the company for the shareholder. Nor do not-for-profits, religious orders, or political parties always act in furtherance of their founding principles. Intuitively, we recognize that their actions are guided by internal goals, which is why we personify them, both legally and in our habits of thought. When talking about “what China wants” or “what General Motors is trying to do,” we are not speaking in metaphors. These organizations act as intelligences that perceive, decide, and act. Like the goals of individual humans, the goals of organizations are complex and often self-contradictory, but they are true goals in the sense that they direct action. Those goals depend somewhat on the goals of the people within the organization, but they are not identical.”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
The question both Wiener and Hillis present us with is how do we move from our current path, one inhabited with unaligned hybrid superintelligences, to one where the hybrid superintelligences are aligned with the interest of human freedom, human love, and human intelligence.
This may simply not be possible. Once control has been ceded from individual humans or groups of humans to hybrid superintelligences, we may never recover it.
Finally, the picture is unfortunately worse than this. The AI community is currently building increasingly capable machine intelligence that will be implemented by machine agents. These machine intelligences, as they become increasingly capable will create a further imbalance between humans and their abilities to influence the behavior of both hybrid superintelligences and increasingly the intelligent machine agents themselves.
It is here that I share Huxley’s concern as he observed the beginning of some of these treacherous trends at the end of A Brave New World Revisted. Huxley cries out:
“Meanwhile there is still some freedom left in the world. Many young people, it is true, do not seem to value freedom. But some of us still believe that, without freedom, human beings cannot become fully human and that freedom is therefore supremely valuable. Perhaps the forces that now menace freedom are too strong to be resisted for very long. It is still our duty to do whatever we can to resist them.”
— Brave New World Revisited by Aldous Huxley
Resist we shall.
****
This has been Chapter 1: Machine Agents, Hybrid Superintelligences, and The Loss of Human Control. This is the first chapter in the ongoing sequence How Humanity Lost Control and Humans Lost Liberty: From Our Brave New World to Analogia.
For chapter 2, before turning to Our Brave New World, I want to start with an updated and abridged presentation of Olaf Stapledon’s “The Other Earth.” Enjoy.
Machine Agents, Hybrid Superintelligences, and The Loss of Human Control (Chapter 1)
“And everybody knows that the Plague is coming
Everybody knows that it’s moving fast
Everybody knows that the naked man and woman
Are just a shining artifact of the past
Everybody knows the scene is dead
But there’s gonna be a meter on your bed
That will disclose
What everybody knows
And everybody knows that you’re in trouble
Everybody knows what you’ve been through
From the bloody cross on top of Calvary
To the beach of Malibu
Everybody knows it’s coming apart
Take one last look at this Sacred Heart
Before it blows
And everybody knows”
-Leonard Cohen, Everybody Knows
Here, at the very beginning of our story, I need to highlight two seemingly boring and mundane points. These points are:
1. Large organizations are immensely powerful in modern society
2. They exhibit their own emergent behavior
These are my starting claims. I assume these as obvious points upon brief reflection. One so obvious as to be commonplace in our language. For example, these claims apply to the large organizations that constitute collective “Big Government”, “Big Business”, and “Big Science.” We speak and write as if this is a generally accepted fact about our world. We say things like “The US did this for that reason,” “Facebook (Meta?) did that to maximize its profit,” “The National Science Foundation decided to fund research for the benefits it perceived over and above the potential costs.” These two claims, I think, all on their own, should cause us pause for concern when we consider the sheer size and reach of our modern organizations (something we’ll return to in a future post). Given their size, complexity, and emergent behavior, these increasingly powerful organizations are also demonstrably unaligned with the general interests of human freedom, human intelligence, and human love.
These two claims need to be taken together with one additional claim, a third important point for our story:
3. These large organizations systematically deploy machine agents to accomplish their goals
These three claims, I argue, point to the current existence and power of superintelligent hybrids. Superintelligences that are already beyond human control. These superintelligences are the large hybrid organizations of people and machines. These hybrid entities are currently more intelligent, they have more memory and more compute among other capabilities and capacities, than any human or human organization had before the incorporation of machine agents.
It is these superintelligent hybrids, with their coupling of new forms of increasingly intelligent and capable machine behavior that may have the most dramatic impact in directing the course of humanity’s future. This is a troubling conclusion, one that I believe should be a major concern of those concerned with human dignity and particularly those actively influencing AI policy, research, development, and deployment. These concerns are playing out now, not at some hypothetical point in the future. These superintelligences are shaping and manipulating our preferences, desires, and attention, in mass, now. And, they already exceed our ability, as individual humans to fully control them.
It is with all of this in mind, that I argue that we, the humanity we, have lost control over the hybrid superintelligences amongst us. We have not, it does not seem, lost total control. But, we have lost a frighteningly larger share of control than collectively we have internalized.
I will build this argument out further in a future post by visiting with Aldous Huxley’s 1958 observations, in A Brave New World Revisited, in particular with Huxley’s observations that the world suffers from trends towards over-population and over-organization, and that these two trends have led to great advances in authoritarianism and mass manipulation by large organizations in the forms of Big Government and Big Business (and sometimes even Big Science!). Following the description of these challenges, I will argue that another trend, that of over-digitalization, has begun to amplify and exacerbate authoritarianism and mass manipulation. These three challenges of over-population, over-organization, and over-digitalization have already greatly altered the relationships between humans, their minds, their environments, their machines, and their organizations.
However, before turning to Huxley’s observations in further detail, we will first visit with an abridged version of Olaf Stapledon’s The Other Earth and see the great tragedy of the Other Men. It is here that we see a picture of a world so different, yet so strikingly similar, to our own. The story is one of eternal radio bliss, or the pleasure of a tailored “news feed.” At the end of Part I of this sequence, you will meet Analogia. Analogia makes the point well enough, that in our time, the “radio” itself begins to take control.
These initial explorations will set us down our journey to understand what is at stake in our current, mad world. The facts on the ground suggest that superintelligence is here, that it is not perfectly aligned with the interests of humanity, and that we have already lost significant control of it.
The ultimate consequence or dominating end of over-population, over-organization, and over-digitalization may therefore be, I will argue, the death of freedom and with it the great beauties of love, humanity, and even the very utility of intelligence.
As we delve into these global trends, it will be necessary to dive a bit further into the nature of humans, machines, and hybrids. To do this we’ll need to discuss organizations themselves as structures inhabited by complex agents and to compare organizations to organisms in the ways that they are generally constituted. Keeping the focus on organizations and institutions we’ll turn to Max Weber, Herbert Simon, Elinor Ostrom, and others for a picture of human organizations and how they navigate and manipulate the complexities of their environments, distributing authority and consuming resources throughout.
Even by Aldous Huxley’s time we lived in a world dominated by the delegation of much of our private authority to increasingly powerful and large organizations. The most obvious examples still being modern nation states and large multinational corporations. And, I will argue that for most, if not all of us, these organizations now exert even more authority and control over our lives than they did in Huxley’s time.
Over-population is a topic that seems to have been absorbed out of the modern dialogue on global problems. However, I believe it is a mistake not to put this fact out front as a leading cause of over-organization and over-digitalization. In the post on over-population, I will argue that while the global population growth rate itself has slowed, the sheer number of new humans has been a leading cause in straining the capacities of the planet, our nations, and our organizations, all of which has left us with both an increasing need for more organization (leading to over-organization) and (often) increases in suffering throughout society.
From our discussion of over-population and over-organization, we will turn to over-digitalization and the failure of cybernetics, that is, just with over-population and over-organization, we have lost significant control of the process of digitalizing our analog world. The earliest AI pioneers Alan Turing, John von Neumann, Norbert Wiener, and others worried over the questions of governing or retaining control of machines and machine behavior embedded in human society. Their concerns were justified, as we now live in Our Brave New World in which over-digitalization has crashed into over-organization and over-population to create a truly frightening path to Analogia. A world George Dyson describes thus:
“In the fourth epoch of technology, the powers of the continuum will be claimed by machines. The next revolution, as fundamental as when analog components were assembled into digital computers, will be the rise of analog systems over which the dominion of digital programming comes to an end. Nature’s answer to those who seek to control nature through programmable machines is to allow us to build systems whose nature is beyond programmable control.
Nothing is to be gained by resisting the advance of the discrete-state machines, for the ghosts of the continuum will soon return, when the grass is eight inches high in the spring. A world shared with true machine intelligence, proliferating in the wild as opposed to the captive, domesticated versions being marketed today, will bring the new millennium that Wovoka and others predicted, infused by an intelligence that belongs not to us but to Nature as a whole.
The human species, and the human mind, grew up in a world shared with creatures large and small, animated by spirits that some were privileged to communicate with but no one claimed to understand, let alone control. ‘The least particle must be considered as a world full of an infinity of different creatures,’ Leibniz admitted in 1693.
Some see only an empty hat. Others see the full power of the continuum, with the powers of the merely countable infinities becoming small fishes in the rivers while megafauna once again roam the earth.
Not even ashes will remain.”
— Analogia: The Emergence of Technology Beyond Programmable Control by George Dyson
Dyson’s argument, which we will dive further into in a later post, I find compelling.
It seems to me that we find ourselves currently living with the introduction of machine agents into our over-populated, over-organized, and over-digitalized world. We are not at the precipice, we have leapt headfirst over the unknown horizon.
The question that immediately presents itself is: what do we do about this?
Here we will find no simple answers. We will see a complex world that is inhabited by machine intelligence, superintelligences, and digital entities beyond programmable control, that, hopefully, we can use to point us away from worse and world-ending directions and towards directions that encourage human freedom, love, and intelligence.
Now that I have painted some of the general contours for you, here and in the sequence introduction, let us spend a few more moments on the consequences of hybrid superintelligences amongst us mere mortals.
Norbert Wiener, early in the rise of AI, paints well the concern for humans of over-organization and the rise of superintelligent hybrid organizations that dominate humans:
“I have spoken of machines, but not only of machines having brains of brass and thews of iron. When human atoms are knit into an organization in which they are used, not in their full right as responsible human beings, but as cogs and levers and rods, it matters little that their raw material is flesh and blood. What is used as an element in a machine, is in fact an element in the machine. Whether we entrust our decisions to machines of metal, or to those machines of flesh and blood which are bureaus and vast laboratories and armies and corporations, we shall never receive the right answers to our questions unless we ask the right questions. . . . The hour is very late, and the choice of good and evil knocks at our door.—NORBERT WIENER, The Human Use of Human Beings”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
It is with this inspiration from Wiener, that Daniel Hillis, offers some insight into our current situation, one surrounded and dominated by hybrid superintelligences:
“What is now clear, whether or not it was apparent to Wiener, is that these organizational superintelligences are not just made of humans, they are hybrids of humans and the information technologies that allow them to coordinate. Even in Wiener’s time, the “bureaus and vast laboratories and armies and corporations” could not operate without telephones, telegraphs, radios, and tabulating machines. Today they could not operate without networks of computers, databases, and decision support systems. These hybrid intelligences are technologically augmented networks of humans. These artificial intelligences have superhuman powers. They can know more than individual humans; they can sense more; they can make more complicated analyses and more complex plans. They can have vastly more resources and power than any single individual.”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
Hillis continues:
“Although we do not always perceive it, hybrid superintelligences such as nation-states and corporations have their own emergent goals. Although they are built by and for humans, they often act like independent intelligent entities, and their actions are not always aligned with the interests of the people who created them. The state is not always for the citizen, nor the company for the shareholder. Nor do not-for-profits, religious orders, or political parties always act in furtherance of their founding principles. Intuitively, we recognize that their actions are guided by internal goals, which is why we personify them, both legally and in our habits of thought. When talking about “what China wants” or “what General Motors is trying to do,” we are not speaking in metaphors. These organizations act as intelligences that perceive, decide, and act. Like the goals of individual humans, the goals of organizations are complex and often self-contradictory, but they are true goals in the sense that they direct action. Those goals depend somewhat on the goals of the people within the organization, but they are not identical.”
— Possible Minds: Twenty-Five Ways of Looking at AI edited by John Brockman
The question both Wiener and Hillis present us with is how do we move from our current path, one inhabited with unaligned hybrid superintelligences, to one where the hybrid superintelligences are aligned with the interest of human freedom, human love, and human intelligence.
This may simply not be possible. Once control has been ceded from individual humans or groups of humans to hybrid superintelligences, we may never recover it.
Finally, the picture is unfortunately worse than this. The AI community is currently building increasingly capable machine intelligence that will be implemented by machine agents. These machine intelligences, as they become increasingly capable will create a further imbalance between humans and their abilities to influence the behavior of both hybrid superintelligences and increasingly the intelligent machine agents themselves.
It is here that I share Huxley’s concern as he observed the beginning of some of these treacherous trends at the end of A Brave New World Revisted. Huxley cries out:
“Meanwhile there is still some freedom left in the world. Many young people, it is true, do not seem to value freedom. But some of us still believe that, without freedom, human beings cannot become fully human and that freedom is therefore supremely valuable. Perhaps the forces that now menace freedom are too strong to be resisted for very long. It is still our duty to do whatever we can to resist them.”
— Brave New World Revisited by Aldous Huxley
Resist we shall.
****
This has been Chapter 1: Machine Agents, Hybrid Superintelligences, and The Loss of Human Control. This is the first chapter in the ongoing sequence How Humanity Lost Control and Humans Lost Liberty: From Our Brave New World to Analogia.
For chapter 2, before turning to Our Brave New World, I want to start with an updated and abridged presentation of Olaf Stapledon’s “The Other Earth.” Enjoy.
** You can find the Sequence Introduction here**