Working on alignment should actually be done at least because If Anyone Misaligns It, Everyone Dies and at most because it could allow workers to align the AIs against the Intelligence Curse. Anthropic’s cofounders have outright pledged to donate “80% of our wealth”.
Revisiting the original post, I think that it is as eloquent as Agent-5 and could be about as truthless. I expect a post-ASI economy to consist of mining resources and of having the AIs and robots satisfy the desires of those who have the resource rights to pay for them. What I struggle to understand is whether there exists a legal structure to which the AIs can be aligned so that the resource rights became neither distributed in an egalitarian way nor concentrated among the few overlords, but were spread among ~1% of people with no way to shrink the percent further. The two cases against such a structure are the collapse of the Roman Empire and the rise of proletariat. Alas, the latter becomes impossible since the new proletariat is the AIs who we assume to be aligned. The former was due to a civil war and barbarian invasions. But an ASI-ruled world lacks similarly capable adversaries!
Revisiting the original post, I think that it is as eloquent as Agent-5 and could be about as truthless
Thanks for the candor—however I unfortunately don’t get the Agent-5 reference!
Alas, the latter becomes impossible since the new proletariat is the AIs
I like this comparison! A lot of people (on the left and right) will argue against the permanent-underclass by gesturing at the labor struggles; but, contrary to popular belief, the power of the proletariat was not in their being the majority (many revolutions took place in countries where they were indeed the minority, vs. the greater agricultural fraction) but rather their economic centrality and the corresponding power of a strike. In a world where humans are deprived of all economic bargaining power, there is no strike! So this question must first be answered: for how long will humans retain economic importance? This is a separate question from “when will [many] people be immiserated”.
Personally, I don’t think we’ll get there so soon, and I think there will be a significant intermediate period where the latter is true but the former is not. Robotics is likely to remain a difficult challenge for some time, and as such so long as humans are necessary for e.g. manual labor, warfare, nursing, etc., they will still be able to fill the historical role of the proletariat.
You might ask, but then why argue against a ‘permanent underclass’? Certainly, I do think we will see an enlarging of the already-existing underclass. But when I say “significant intermediate period”, I really am thinking on the scale of years, not decades. And the predominant forces of this period will not be that of an ‘economic contraction’, but destabilization.
The former was due to a civil war and barbarian invasions. But an ASI-ruled world lacks similarly capable adversaries!
This is not the right analogy. An ASI-ruled world is one where this strand of socio-economic development has already reached fruition; that would be analogous to the state that medieval Europe reached centuries after the collapse of the west, a stable order constructed out of the ruins. The period I’m concerned about is the time where (to paraphrase Gramsci) “The old world is dying and the new world struggles to be born. [This] is the time of monsters”. More precisely, the period of transition is one where neither the ‘old system’ nor the ‘new system’ are secure in their position, and a variety of developmental forces are in play at once—to take an alternate paraphrase of the same quote, “in this interregnum a great variety of morbid symptoms appear”. If you consider the period where, say, the AI research loop is near-to-closed but humans are still necessary for warfare (even as larger and larger portions of the population are deprived of their income), we can imagine many ‘morbid symptoms’ that could lead to civil war, and which could take the role of “barbarians at the gate”. Terrorism is an easy example! Certainly terrorists are internal to the polity, but so too were the Germanic barbarians a part of the Roman system (their governments, their economies, even their writing systems were all centered around Rome).
To both of your points above, I think my first response is that you are talking about the endgame, while I am trying to talk about the path that would lead us there. Yes, if you speed by and just assume we land in a self-stable ASI society, things look different, but this sort of ‘fast analysis’ is exactly what I want to argue against. The period of transition is far more important than any hypothetical end-state, not least because it comes first.
Alas, the latter becomes impossible since the new proletariat is the AIs who we assume to be aligned
To be honest I don’t think “who we assume to be aligned” is a reasonable assumption to make. Perhaps that’s the root of your disagreement?
Regardless, I do want to ask, to what extent do you disagree with the actual thesis of my post? I.e., is it the case that you do think there will be a permanent underclass, and that the ‘right thing’ to do is for one to ensure one’s own safety by selling labor to our future overlords (at a good price)?
Working on open-source models is the best way to empower terrorists instead of the masses.
Working on alignment should actually be done at least because If Anyone Misaligns It, Everyone Dies and at most because it could allow workers to align the AIs against the Intelligence Curse. Anthropic’s cofounders have outright pledged to donate “80% of our wealth”.
Revisiting the original post, I think that it is as eloquent as Agent-5 and could be about as truthless. I expect a post-ASI economy to consist of mining resources and of having the AIs and robots satisfy the desires of those who have the resource rights to pay for them. What I struggle to understand is whether there exists a legal structure to which the AIs can be aligned so that the resource rights became neither distributed in an egalitarian way nor concentrated among the few overlords, but were spread among ~1% of people with no way to shrink the percent further. The two cases against such a structure are the collapse of the Roman Empire and the rise of proletariat. Alas, the latter becomes impossible since the new proletariat is the AIs who we assume to be aligned. The former was due to a civil war and barbarian invasions. But an ASI-ruled world lacks similarly capable adversaries!
Thanks for the candor—however I unfortunately don’t get the Agent-5 reference!
I like this comparison! A lot of people (on the left and right) will argue against the permanent-underclass by gesturing at the labor struggles; but, contrary to popular belief, the power of the proletariat was not in their being the majority (many revolutions took place in countries where they were indeed the minority, vs. the greater agricultural fraction) but rather their economic centrality and the corresponding power of a strike. In a world where humans are deprived of all economic bargaining power, there is no strike! So this question must first be answered: for how long will humans retain economic importance? This is a separate question from “when will [many] people be immiserated”.
Personally, I don’t think we’ll get there so soon, and I think there will be a significant intermediate period where the latter is true but the former is not. Robotics is likely to remain a difficult challenge for some time, and as such so long as humans are necessary for e.g. manual labor, warfare, nursing, etc., they will still be able to fill the historical role of the proletariat.
You might ask, but then why argue against a ‘permanent underclass’? Certainly, I do think we will see an enlarging of the already-existing underclass. But when I say “significant intermediate period”, I really am thinking on the scale of years, not decades. And the predominant forces of this period will not be that of an ‘economic contraction’, but destabilization.
This is not the right analogy. An ASI-ruled world is one where this strand of socio-economic development has already reached fruition; that would be analogous to the state that medieval Europe reached centuries after the collapse of the west, a stable order constructed out of the ruins. The period I’m concerned about is the time where (to paraphrase Gramsci) “The old world is dying and the new world struggles to be born. [This] is the time of monsters”. More precisely, the period of transition is one where neither the ‘old system’ nor the ‘new system’ are secure in their position, and a variety of developmental forces are in play at once—to take an alternate paraphrase of the same quote, “in this interregnum a great variety of morbid symptoms appear”. If you consider the period where, say, the AI research loop is near-to-closed but humans are still necessary for warfare (even as larger and larger portions of the population are deprived of their income), we can imagine many ‘morbid symptoms’ that could lead to civil war, and which could take the role of “barbarians at the gate”. Terrorism is an easy example! Certainly terrorists are internal to the polity, but so too were the Germanic barbarians a part of the Roman system (their governments, their economies, even their writing systems were all centered around Rome).
To both of your points above, I think my first response is that you are talking about the endgame, while I am trying to talk about the path that would lead us there. Yes, if you speed by and just assume we land in a self-stable ASI society, things look different, but this sort of ‘fast analysis’ is exactly what I want to argue against. The period of transition is far more important than any hypothetical end-state, not least because it comes first.
To be honest I don’t think “who we assume to be aligned” is a reasonable assumption to make. Perhaps that’s the root of your disagreement?
Regardless, I do want to ask, to what extent do you disagree with the actual thesis of my post? I.e., is it the case that you do think there will be a permanent underclass, and that the ‘right thing’ to do is for one to ensure one’s own safety by selling labor to our future overlords (at a good price)?