You’re going to have a tough time with Dostoyevsky if you think about it as “worshipping religion”. It might help to remember that he was also quite conversant with arguments for atheism, e.g. as he has Ivan espouse in Brothers Karamazov. Not to say I think you should believe like he does, but that he’s not just putting this stuff forward out of blind ignorance.
Marcus Plutowski
Karma: 119
That’s fair! On reflection I’m used to dealing with the opposite issue given some of the circles I’m in, so I think my instincts are tuned in the wrong direction and your framing was probably the right choice.
I’m surprised by your confusion at Pope Leo’s juxtaposition of Babel and Nehemiah, as the metaphor seemed rather clear to me, while the ‘candidate interpretations’ you provided feel very modern and out of place. It’s possible that this comes down to differences in exegetical tradition between Christian and Jewish schools of thought, in which case I’m probably not properly equipped to bridge that gap. Assuming not, however, I’ll give a stab at my understanding.
Starting from the text: the builders of Babel say
Then they said, “Come, let us build ourselves a city, and a tower with its top in the heavens, and let us make a name for ourselves; otherwise we shall be scattered abroad upon the face of the whole earth.” The Lord came down to see the city and the tower, which mortals had built. And the Lord said, “Look, they are one people, and they have all one language; and this is only the beginning of what they will do; nothing that they propose to do will now be impossible for them. Come, let us go down, and confuse their language there, so that they will not understand one another’s speech.
There are a few things I want to point out. First, the intentions of the builders are directly self-glorifying: to make a name for themselves, to aggrandize themselves through great works. Secondly, the motivation for the construction is a sense of vulnerability: they fear the prospect of being ‘scattered abroad’, and seek to create an anchoring-point to hold their population in one place. Yet this is directly opposed to God’s prior command to “Be fruitful and multiply, and fill the earth”, and (in Catholic thought, at least) this of command is not made out of pique, but because it is the proper orientation for human life. That is to say: God has laid out a plan for mankind; mankind is afraid of what will happen if they let that plan unfold, because doing so would mean giving up control; they seek to forestall God’s plan through their own power; in doing so, they harm themselves. God’s subsequent intervention merely restores the proper order of the world.
The parallels to AI seem clear. Those who seek to build AI do so largely out of a combination fear, greed, and pride (also greed, but that’s perhaps less relevant for the Babel analogy so I’ll ignore it for now). The leaders of the labs are afraid of China (or another lab) winning a ‘race to AGI’, and all harbor some level of a prideful desire to see their own worldview ‘conquer the lightcone’. The engineers on the ground have FOMO plus the obvious vainglory of working at a cool lab and being acclaimed by their peers. More than either of those though, everyone is afraid of death, and many AI boosters claim that the risk of AI is justified either because it’ll fix death or because we’ll destroy ourselves anyways if we don’t build AGI soon. The Catholic view, of course, is that mankind cannot actually destroy ourselves (because we have to make it to the Second Coming), death is a natural part of life (and indeed how you get to heaven), and that God’s plan should+will determine the future of the lightcone. So AI is the modern Babel because it is man’s attempt to defy God’s plan through a misplaced sense of confidence in our own capabilities and our own sense of what the world ‘should be like’.
Out of curiosity, how much did training the Mistral model to the final checkpoint cost you (approx)?
Another, separate point:
Eventually, there will be tension between a blanket ban on lethal autonomous weapons and the Church’s concern for alleviating objective suffering. I hope the Church engages with the nuanced question of what principles can minimize actual harm to innocents while preserving ultimate human moral responsibility for uses of force.
I don’t think that there’s going to be much of a conflict here. What you’re describing is a conflict between a utilitarian imperative (minimize suffering / bad outcomes) and one rooted in virtue ethics (or even divine law?). In Catholic philosophy the latter tends to win out. Very rarely do you see Church teachings suggest the sacrifice of principles in favor of practical concerns. An instructive example is the case of abortion, say in the case of a mother without the means to take care of her child, with the likely outcome of a birth being the child living a life of poverty. Many will argue that the ‘misery-minimizing’ choice would be to allow the mother to conduct an abortion (and some rather less savory types, i.e. eugenicists, will even argue that it should be made mandatory). But obviously Catholic teaching does not permit this; abortion can be permitted when it occurs as a side-effect of medical treatment for the mother, but certainly not to ‘alleviate objective suffering’. The Church would instead suggest that people do something to improve the situation of the mother and her child, as it does through various charity networks.
It might be clarifying to note that the Church does not consider suffering to be the ‘worst thing’ (though it is certainly bad); sin is the worst thing, and the Church generally advises people to suffer now rather than sin and (in their eyes) risk eternal damnation later. Now of course, it’s not clear a priori that “using lethal autonomous weapons” is necessarily a sin, insofar as it’s not called out as such in the Bible or anything like that. But Catholic teaching has been moving steadily towards a broad-spectrum rejection of violence, one which harkens back earlier centuries before just war doctrine and other carveouts for the ‘licit’ deployment of force emerged. There was a time when e.g. Byzantine soldiers, fighting on behalf of a Christian emperor, would rush to priests after battle to receive absolution, just in case their killing (in warfare) was against the will of God. Those clarifications that did develop in later centuries, like the Thomist concept of just war, all rely on a carefully considered use of force mediated by human reason. So I would expect that, by default, deployment of violence in a capacity divorced from human reason would be considered sinful, because violence, by default, is sinful. The surprising thing isn’t that the Church considers new forms of warfare to be sinful in ways that old ones are not, it’s that new forms of violence are presumed (by governments, if not the general population) to be OK and only banned if exigent circumstances demand it.
A further analogy would be to consider the use of normal, non-autonomous warfare in pursuit of a (purportedly) noble goal—e.g. to invade a poor and corrupt nation, overthrow its leaders, and impose a more efficient system that can actually provide modern medical care to the general population. This would arguably reduce general suffering, but the Church explicitly does not endorse this. I expect the same rationale to apply to the use of autonomous warfare, even in situations like you describe.
Glad to see this post; while I’m sure most Lesswrong posters are at least aware of the encyclical, I expect most have not taken the time to read it. I think that in terms of defining the moral foundations for an anti-AI movement, it’s leagues ahead of other entrants: however you feel about “traditional morality”, the language and structure thereof are intuitively familiar to everyday people in a way that e.g. the lightcone is not. Obviously both rhetorical approaches have their tradeoffs, but when we’re talking about building a political coalition, the former is essential—and, I think, greatly underserved in the AI safety space.
One thing I wanted to poke at, though:Sidebar: many Americans misunderstand Catholic references to “the common good.” The Church supports healthy private enterprise—this term doesn’t imply communist collectivism. … The Church supports private property and free enterprise as good ways to promote the common good, but insists that these be implemented with a recognition that all people share in a basic right to a just portion of these resources.
You’re not wrong, but I do think that this presentation might be somewhat misleading, or indicate a misunderstanding of what the Church teaches and why. Rerum Novarum explores this in more detail, but at a high level the idea is that private property is important not for its own sake, but insofar as it bolsters human dignity (which is important for its own sake). This means that there is a difference in how, for example, capital equipment and personal goods might be considered under a Catholic social framework. We can see that the latter clearly bolster human dignity: it is more dignified to own one’s own toothbrush, one’s own home, even one’s own retirement savings, than to rely on the state or some other actor (who might take them away). But the same cannot so directly be said about the former (e.g. ownership of a factory, or of the town mill). While Pope Leo XIII was certainly no socialist, he was also certainly anti-capitalist, and introduced the concept of Subsidiarity which later Popes would build on (e.g. in Quadragesimo anno). The essential idea is that local/‘natural’ groupings of people (families and local communities especially) should not be subordinated to a remote authority except when necessary. Obviously this applies to central planning, but it applies equally well to the influence and sway of large (and today, transnational) corporations. Pope Pius XI developed this into a loose theory of ‘distributism’, which really looks a lot more like medieval guild organization than either capitalism or socialism.
Perhaps that was a long way of saying that the Church certainly supports private property of certain kinds, but otherwise its support is largely situational, and not a primary focus. The first priority is always the dignity of the individual.
That’s not true. We have evidence that after agriculture, individuals were materially less well off, eating not only less food but lower quality food. Even once populations reached their new carrying capacity, this effect persisted, even though (just?) as many children were dying in infancy.
That’s of very little comfort to presently-existing humans who are staring down the barrel of QoL reductions.
it’s evil and it won’t even work
Great condensation of my central point!
The next question is what counts as building a raft for everyone. Political action in favor of regulation? (I’d say yes.) Working on alignment at big labs? (I’d say no.) Working on open models to tilt the balance of AI power toward the masses? (I’d say yes.)
Agree on the first two points; working on alignment at the big labs seems like a losing game even in the direct sense, and from what I hear tends to lead to accidentally-accelerating-capabilities regardless. I’m not so sure about working on open models, though I don’t oppose it: I just think that political-economic forces will ensure their development regardless, as race dynamics between China and the US have already kicked in.
Beyond (and in support of) political action in favor of regulation, I think that popular mobilization is a necessary prerequisite to any ‘happy ending’. While direct electoral accountability is in practice limited, sufficiently large and angry crowds have a power all their own (c.f. the achievements of the civil rights movement).
Revisiting the original post, I think that it is as eloquent as Agent-5 and could be about as truthless
Thanks for the candor—however I unfortunately don’t get the Agent-5 reference!
Alas, the latter becomes impossible since the new proletariat is the AIs
I like this comparison! A lot of people (on the left and right) will argue against the permanent-underclass by gesturing at the labor struggles; but, contrary to popular belief, the power of the proletariat was not in their being the majority (many revolutions took place in countries where they were indeed the minority, vs. the greater agricultural fraction) but rather their economic centrality and the corresponding power of a strike. In a world where humans are deprived of all economic bargaining power, there is no strike! So this question must first be answered: for how long will humans retain economic importance? This is a separate question from “when will [many] people be immiserated”.
Personally, I don’t think we’ll get there so soon, and I think there will be a significant intermediate period where the latter is true but the former is not. Robotics is likely to remain a difficult challenge for some time, and as such so long as humans are necessary for e.g. manual labor, warfare, nursing, etc., they will still be able to fill the historical role of the proletariat.
You might ask, but then why argue against a ‘permanent underclass’? Certainly, I do think we will see an enlarging of the already-existing underclass. But when I say “significant intermediate period”, I really am thinking on the scale of years, not decades. And the predominant forces of this period will not be that of an ‘economic contraction’, but destabilization.
The former was due to a civil war and barbarian invasions. But an ASI-ruled world lacks similarly capable adversaries!
This is not the right analogy. An ASI-ruled world is one where this strand of socio-economic development has already reached fruition; that would be analogous to the state that medieval Europe reached centuries after the collapse of the west, a stable order constructed out of the ruins. The period I’m concerned about is the time where (to paraphrase Gramsci) “The old world is dying and the new world struggles to be born. [This] is the time of monsters”. More precisely, the period of transition is one where neither the ‘old system’ nor the ‘new system’ are secure in their position, and a variety of developmental forces are in play at once—to take an alternate paraphrase of the same quote, “in this interregnum a great variety of morbid symptoms appear”. If you consider the period where, say, the AI research loop is near-to-closed but humans are still necessary for warfare (even as larger and larger portions of the population are deprived of their income), we can imagine many ‘morbid symptoms’ that could lead to civil war, and which could take the role of “barbarians at the gate”. Terrorism is an easy example! Certainly terrorists are internal to the polity, but so too were the Germanic barbarians a part of the Roman system (their governments, their economies, even their writing systems were all centered around Rome).
To both of your points above, I think my first response is that you are talking about the endgame, while I am trying to talk about the path that would lead us there. Yes, if you speed by and just assume we land in a self-stable ASI society, things look different, but this sort of ‘fast analysis’ is exactly what I want to argue against. The period of transition is far more important than any hypothetical end-state, not least because it comes first.
Alas, the latter becomes impossible since the new proletariat is the AIs who we assume to be aligned
To be honest I don’t think “who we assume to be aligned” is a reasonable assumption to make. Perhaps that’s the root of your disagreement?
Regardless, I do want to ask, to what extent do you disagree with the actual thesis of my post? I.e., is it the case that you do think there will be a permanent underclass, and that the ‘right thing’ to do is for one to ensure one’s own safety by selling labor to our future overlords (at a good price)?
Against the “Permanent” Underclass
What exactly do you mean by “Marxist policies”? Do you consider “No Child Left Behind” to be a Marxist policy, or Obama to be a Marxist?
If we valued democracy, but did not sacralize it, then we would treat ensuring democracy as a mundane engineering problem, and would create better policies.
This seems like a reach. For example, there was clearly a time when democracy was not, in itself, sacred—a transitional period of several hundred years in e.g. Great Britain where ideas like “personal liberty”, “private property”, and “consent of the governed” came to be sacralized, followed by democracy itself. In some places, this “engineering problem” was indeed addressed in a way that produced better policies, e.g. Great Britain. In others, e.g. Russia, this did not happen: the Tsar instead “solved” the problem by using his established power to rig the Duma elections and pack it with politically compatible lackeys.
Or alternatively, consider another example from the same-ish reference class, which we as a society do value but do not sacralize: minimizing the government’s fiscal deficit. That “engineering problem” has gone rather unsolved for some time now.
I don’t know about the last part:
> Or, if you limit progress to my personal understanding of philosophy? Then I can just read stuff and get better than what I would have thought up myself. Same strategy, in some sense.I think (ha!) that acquiring an understanding of philosophy does require ‘thinking’ in a way that some other fields do not. For example, I can get a reasonably good understanding of the history of WW2 by memorizing a sufficiently long list of facts and being able to reproduce them exactly. Obviously thinking, extrapolating, making inferences would help to improve one’s understanding further—but there’s some sense in which a large percentage of the ‘content’ of a historical understanding of WW2 is indeed contained in the bare facts as they happened. This makes sense as history is indeed first and foremost a study of “what happened?”, and secondarily a study of “why”, “how”, etc. I don’t think the same applies to philosophy. To take an example from my own experience, memorizing Deleuze & Guattari’s definition of a “Rhizome” will do little to help you apply it. I suspect that even memorizing the whole of ATP would do little in that regard! So with regards to improving your “personal understanding” of philosophy, I do think that you need to think, even when given the results of past philosophers to go off of. In some sense I would think of those results as a shortcut towards where to think, what paths of deliberation are productive to go down. So while you would certainly benefit more from reading those alien blogposts than by thinking on your own, you would need to balance (in some proportion -- 50⁄50, 90⁄10, I don’t know) thinking and reading to actually maximize your own understanding.
To be clear, I generally disagree with most varieties of the simulation hypothesis. But nevertheless I do think that this question in particular has some good answers—a lot of which more-or-less reduce to ‘forecasting outcomes via an agent-based simulation’. Unfortunately that family of explanations wouldn’t give you much in the way of understanding the simulators, as it could be anything from “simulate outcomes of X choice in a copy of [the simulator’s] world” (which would tell you that they’d be very similar to us) to “consider this strange hypothetical where apes evolved intelligence, for purely academic reasons” (which would tell you much less). Alternatively there are some which veer far away from that, to e.g. entertainment, or pleasure of some kind (e.g. an elaborate form of pornography?), or even some kind of long-form training algorithm, and I do think that in those cases you can infer more about the motivations of the simulator.
I don’t think that’s necessarily true. I go off topic with coworkers when it feels appropriate, and if anything it can be a nice break from grinding on whatever problem.
Is that so bad? The rational use of irrational symbols has proven highly effective in the past. Whatever it takes to survive, is worth considering.
It seems to me that elites are elite because they are simply the best at doing whatever it takes to maintain their power
This is true, but it’s clearly conditioned on the currently-existing socioeconomic system. The aristocracy of post-feudal Europe did quite a good job of holding on to power for quite a long time, but over the course of just a few decades (1880-1920, or more expansively the whole century from 1850 to 1950) lost most-all of it to the up-and-coming bourgeoisie. This is despite them holding all the usual suspects for “tools to maintain power at all costs”:
political repression
control of the military
heavily-biased elections and political systems more broadly
ownership of most land in most countries
control of state institutions
Yet that was still swept away once, depending on the particular country, either the internal structural pressures grew too great for the aristocracy to hold on any longer (e.g. France, the UK), or a misstep in the face of external pressure broke their ability to maintain the above systems of suppression (e.g. Germany, Austria, Russia).
any replacement elite would have to do similar things or itself be replaced
Considering again the above: after the bourgeoisie replaced the aristocracy as the new ruling class, it’s easy to see that they began to do very different things, both in terms of government policy and in terms of how they maintained power thereafter: beyond the cursory “they are rich and spend lots of money” and “they try to maintain power”, there’s very little similarity between the two. The way they maintain power (e.g. electioneering, party capture, money-in-politics, rotating heads of government but all beholden to the markets), what they do with the power (opening up international markets, privatizing industries, lowering taxes), and the people who benefit the most (business-owners, large financiers) are all totally different.
If it were possible for the majority to govern society for their own benefit from the bottom up
You could say the same in 1850, after the failure of the revolutions of 1848: “if it were possible for an elected parliament to govern society without a King to manage them, we’d be living in a liberal-capitalist utopia already”. I think history shows that there is often significant hysteresis and high activation costs to societal change, so the fact that we haven’t already disempowered the current ruling class tells us rather little about whether it’s generally possible to do so.
I struggle to understand what the disagreement with this comment is, I’d appreciate it if someone could fill me in. What other logical response would there be if people in the US government came to accept Anthropic’s messaging on the issue? Certainly we can’t expect them to try and reach out to China for collaboration on the topic, given the policies of this government that we’ve seen thus far