I don’t believe the standard story of the resource curse.
What do you think is the correct story for the resource curse?
I find the scenario implausible.
This is not a scenario, it is a class of concerns about the balance of power and economic misalignment that we expect to be a force in many specific scenarios. My actual scenario is here.
The “social-freeze and mass-unemployment” narrative seems to assume that AI progress will halt exactly at the point where AI can do every job but is still somehow not dangerous.
We do not assume AI progress halts at that point. We say several times that we expect AIs to keep improving. They will take the jobs, and they will keep on improving beyond that. The jobs do not come back if the AI gets even smarter. We also have an entire section dedicated to mitigating the risks of AIs that are dangerous, because we believe that is a real and important threat.
More directly, full automation of the economy would mean that AI can perform every task in companies already capable of creating military, chemical, or biological threats. If the entire economy is automated, AI must already be dangerously capable.
Exactly!
I expect reality to be much more dynamic, with many parties simultaneously pushing for ever-smarter AI while understanding very little about its internals.
“Reality will be dynamic, with many parties simultaneously pushing for ever-smarter AI [and their own power & benefit] while understanding very little about [AI] internals [or long-term societal consequences]” is something I think we both agree with.
I expect that approaching superintelligence without any deeper understanding of the internal cognition this way will give us systems that we cannot control and that will get rid of us. For these reasons, I have trouble worrying about job replacement.
If we hit misaligned superintelligence in 2027 and all die as a result, then job replacement, long-run trends of gradual disempowerment, and the increased chances of human coup risks indeed do not come to pass. However, if we don’t hit misaligned superintelligence immediately, and instead some humans pull a coup with the AIs, or the advanced AIs obsolete humans very quickly (very plausible if you think AI progress will be fast!) and the world is now states battling against each other with increasingly dangerous AIs while feeling little need to care for collateral damage to humans, then it sure will have been a low dignity move from humanity if literally no one worked on those threat models!
You also seem to avoid mentioning the extinction risk in this text.
The audience is primarily not LessWrong, and the arguments for working on alignment & hardening go through based on merely catastrophic risks (which we do mention many times). Also, the series is already enough of an everything-bagel as it is.
Following up with some resource curse literature that understands the problem as incentive misalignment:
On how state revenue sources shape institutional development and incentives, Karl (1997) writes,
“Thus the fate of oil-exporting countries must be understood in a context in which economies shape institutions and, in turn, are shaped by them. Specific modes of economic development, adapted in a concrete institutional setting, gradually transform political and social institutions in a manner that subsequently encourages or discourages productive outcomes. Because the causal arrow between economic development and institutional change constantly runs in both directions, the accumulated outcomes give form to divergent long-run national trajectories. Viewed in this vein, economic effects like the Dutch Disease become outcomes of particular institutional arrangements and not simply causes of economic decline. This deeper explanation is revealed in the relentless interaction between a mode of economic development and the political and social institutions it fosters.
[...]
How are frameworks for decision-making created and reproduced in late-developing countries? I argue that determining the “structuring principle” for these countries—that is, the appropriate starting point for identifying how ranges of choice are constructed—should begin with their leading sector. This means examining the export dependence that molds their economies, societies, and state institutional capacities, and that, in turn, is either reinforced or transformed by them. My effort to understand this set of interactions begins with differentiating the asset specificity, tax structure, and other features inherent in the exploitation of one particular commodity, petroleum. It terminates by examining the state, where the impact of particular economic models and
A central corollary of this argument is that countries dependent on the same export activity are likely to display significant similarities in the capacity of their states to guide development. In other words, countries dependent on mining should share certain properties of “stateness,” especially their framework for decision-making and range of choice, even though their actual institutions are quite different in virtually all other respects. This should be true unless significant state building has occurred prior to the introduction of the export activity.
The specific mechanism for the creation of this institutional sameness lies in the origin of state revenues. It matters whether a state relies on taxes from extractive activities, agricultural production, foreign aid, remittances, or international borrowing because these different sources of revenues, whatever their relative economic merits or social import, have a powerful (and quite different) impact on the state’s institutional development and its abilities to employ personnel, subsidize social and economic programs, create new organizations, and direct the activities of private interests. Simply stated, the revenues a state collects, how it collects them, and the uses to which it puts them define its nature. Thus it should not be surprising that states dependent on the same revenue source resemble each other in specific ways (and consequently so do the decisions made by their leaders).”
I’d note that Karl’s argument has nearly 5,000 citations and is one of the most common (if not the dominant) explanations of the resource curse.
“Oil can turn a gatekeeper state into a caricature of itself. Unlike agriculture, which involves vast numbers of people in the production and marketing of exports, oil requires little labor, and much of it from foreigners. It also entails relationships between the few global firms capable of extracting it and the state rulers who collect the rents. It defines a spigot economy: whoever controls access to the tap, collects the rent.”
On the importance of taxing citizens to state development, Centeno (1997) notes:
“The key to the relationship between war and state making in Western Europe is what Finer (1975) calls the “extraction-coercion” cycle. [...] For the “extraction-coercion cycle” to begin, the relevant states must not have alternative sources of financing while the domestic economy must be capable of sustaining the new fiscal and bureaucratic growth. Conflict-induced extraction will only occur if easier options are not available. Even then, the relevant societies might not be able to produce enough surplus to make the effort productive. Thus, for example, the availability of Latin American silver and the willingness of bankers to risk massive sums freed the Spanish Hapsburgs from imposing greater fiscal control over their provinces as a means to pay for their wars. Conversely, the relative scarcity of such external supports drove the expansion of the early English state.”
On how non-taxation revenue inhibited state development in Latin America, and therefore did not follow Tilley’s pattern of “war making states”, Centeno (1997) argues:
“As in the European cases, war produced immediate deficits, but with one prominent exception, the Latin American states did not respond to these with increased extractions, at least not in the form of domestic taxes. [...] If they could not borrow on international markets (as was the case from roughly 1830 to 1870), Latin American states could sell access to a commodity. Guano allowed Peru to become what Shane Hunt (1973) has called a “rentier state.” The availability of guano revenues retarded the development of the state by allowing it to exist without the remotest contact with the society on which it rested and without having to institute a more efficient administrative machine. Guano did allow the removal of the regressive contribucion (in 1855), but it also permitted the state to avoid modernizing its fiscal structure while borrowing large amounts of money. A contemporary British observer (Markham 1883, p. 37; my emphasis) noted that “a wise government would have treated this source of revenues as temporary and extraordinary. The Peruvians looked upon it as if it was permanent, abolishing other taxes, and recklessly increasing expenditure.” Much like the guano bonanza in the Peruvian case, the conquest of nitrate territories allowed the Chilean state to expand without having to “penetrate” its society and confront the rampant inequality (Loveman 1979, p. 169; Sater 1986, p. 227). By 1900, nitrate and iodine were accounting for 50% of Chilean revenues and 14% of GDP (Mamalakis 1977, pp. 19–21; Sater 1986, p. 275).”
Happy to cite some more of the literature if it’s helpful.
What do you think is the correct story for the resource curse?
This is not a scenario, it is a class of concerns about the balance of power and economic misalignment that we expect to be a force in many specific scenarios. My actual scenario is here.
We do not assume AI progress halts at that point. We say several times that we expect AIs to keep improving. They will take the jobs, and they will keep on improving beyond that. The jobs do not come back if the AI gets even smarter. We also have an entire section dedicated to mitigating the risks of AIs that are dangerous, because we believe that is a real and important threat.
Exactly!
“Reality will be dynamic, with many parties simultaneously pushing for ever-smarter AI [and their own power & benefit] while understanding very little about [AI] internals [or long-term societal consequences]” is something I think we both agree with.
If we hit misaligned superintelligence in 2027 and all die as a result, then job replacement, long-run trends of gradual disempowerment, and the increased chances of human coup risks indeed do not come to pass. However, if we don’t hit misaligned superintelligence immediately, and instead some humans pull a coup with the AIs, or the advanced AIs obsolete humans very quickly (very plausible if you think AI progress will be fast!) and the world is now states battling against each other with increasingly dangerous AIs while feeling little need to care for collateral damage to humans, then it sure will have been a low dignity move from humanity if literally no one worked on those threat models!
The audience is primarily not LessWrong, and the arguments for working on alignment & hardening go through based on merely catastrophic risks (which we do mention many times). Also, the series is already enough of an everything-bagel as it is.
Following up with some resource curse literature that understands the problem as incentive misalignment:
On how state revenue sources shape institutional development and incentives, Karl (1997) writes,
I’d note that Karl’s argument has nearly 5,000 citations and is one of the most common (if not the dominant) explanations of the resource curse.
From Cooper (2002) Chapter 7:
On the importance of taxing citizens to state development, Centeno (1997) notes:
On how non-taxation revenue inhibited state development in Latin America, and therefore did not follow Tilley’s pattern of “war making states”, Centeno (1997) argues:
Happy to cite some more of the literature if it’s helpful.