The Hanson model of sacralization ignores what I think are pretty obvious upsides.
I would contend:
If democracy were not sacred, and treated as one tradeoff amongst others, nearly every elected government in command of bureaucrats and every military organization would find strong reasons to exercise control directly (and to expect their opponents to move first if they did not.)
If education were not sacred, educators would put in much less effort for much higher wages. There would be much less parental pressure for low-performing students to stay in school or high-performing students not to cheat.
If antiracism were not sacred, it would be very easy to build political coalitions around excluding some group from the protections of society.
If religion were not sacralized the associated practices (phrasing it this way to avoid tautology) would disappear pretty quickly.
Now maybe one’s attitude is that if there were no religion (or for that matter democracy, education, antiracism, whatever) then so much the better. But my intuition is largely that most of these things simply don’t survive at all without the spontaneous contribution to public goods, and social fear of contributing to public bads, that sacralization encourages; if you like rule of law, universal literacy, and so on, they disappear pretty quickly. My model is that especially in art and research, but probably also in many other spheres especially education and healthcare, most production only happens because people really care about putting in good work rather than hack work.
Hanson should be smart enough to see this, he just doesn’t like what is currently sacralized.
Of course it’s possible these upsides don’t apply to AIs, but my guess is that without something that’s the equivalent of sacred devotion to the survival of the human race, we do not get that thing.
I disagree with all those contentions. I think you are jumping too quickly from “society values about X” to “society sacralizes X”. I would say that society is much better at achieving things that it values non-sacrally than sacrally.
For example, you write:
If democracy were not sacred, and treated as one tradeoff amongst others, nearly every elected government in command of bureaucrats and every military organization would find strong reasons to exercise control directly (and to expect their opponents to move first if they did not.)”
If we valued democracy, but did not sacralize it, then we would treat ensuring democracy as a mundane engineering problem, and would create better policies.
If we valued democracy, but did not sacralize it, then we would treat ensuring democracy as a mundane engineering problem, and would create better policies.
This seems like a reach. For example, there was clearly a time when democracy was not, in itself, sacred—a transitional period of several hundred years in e.g. Great Britain where ideas like “personal liberty”, “private property”, and “consent of the governed” came to be sacralized, followed by democracy itself. In some places, this “engineering problem” was indeed addressed in a way that produced better policies, e.g. Great Britain. In others, e.g. Russia, this did not happen: the Tsar instead “solved” the problem by using his established power to rig the Duma elections and pack it with politically compatible lackeys.
Or alternatively, consider another example from the same-ish reference class, which we as a society do value but do not sacralize: minimizing the government’s fiscal deficit. That “engineering problem” has gone rather unsolved for some time now.
The Hanson model of sacralization ignores what I think are pretty obvious upsides.
I would contend:
If democracy were not sacred, and treated as one tradeoff amongst others, nearly every elected government in command of bureaucrats and every military organization would find strong reasons to exercise control directly (and to expect their opponents to move first if they did not.)
If education were not sacred, educators would put in much less effort for much higher wages. There would be much less parental pressure for low-performing students to stay in school or high-performing students not to cheat.
If antiracism were not sacred, it would be very easy to build political coalitions around excluding some group from the protections of society.
If religion were not sacralized the associated practices (phrasing it this way to avoid tautology) would disappear pretty quickly.
Now maybe one’s attitude is that if there were no religion (or for that matter democracy, education, antiracism, whatever) then so much the better. But my intuition is largely that most of these things simply don’t survive at all without the spontaneous contribution to public goods, and social fear of contributing to public bads, that sacralization encourages; if you like rule of law, universal literacy, and so on, they disappear pretty quickly. My model is that especially in art and research, but probably also in many other spheres especially education and healthcare, most production only happens because people really care about putting in good work rather than hack work.
Hanson should be smart enough to see this, he just doesn’t like what is currently sacralized.
Of course it’s possible these upsides don’t apply to AIs, but my guess is that without something that’s the equivalent of sacred devotion to the survival of the human race, we do not get that thing.
I disagree with all those contentions. I think you are jumping too quickly from “society values about X” to “society sacralizes X”. I would say that society is much better at achieving things that it values non-sacrally than sacrally.
For example, you write:
If we valued democracy, but did not sacralize it, then we would treat ensuring democracy as a mundane engineering problem, and would create better policies.
This seems like a reach. For example, there was clearly a time when democracy was not, in itself, sacred—a transitional period of several hundred years in e.g. Great Britain where ideas like “personal liberty”, “private property”, and “consent of the governed” came to be sacralized, followed by democracy itself. In some places, this “engineering problem” was indeed addressed in a way that produced better policies, e.g. Great Britain. In others, e.g. Russia, this did not happen: the Tsar instead “solved” the problem by using his established power to rig the Duma elections and pack it with politically compatible lackeys.
Or alternatively, consider another example from the same-ish reference class, which we as a society do value but do not sacralize: minimizing the government’s fiscal deficit. That “engineering problem” has gone rather unsolved for some time now.