Fair enough. As evolutionary ethics tells us, human moral/social instincts evolved in an environment that generally encouraged compromise within a tribe (or perhaps small group of currently-allied tribes). So humans often may act in social-compact-like ways without necessarily consciously thinking through all the reasons why this is generally a good (i.e. evolutionarily adaptive) idea. So I guess I’m used to thinking of a social compact as not necessarily an entirely conscious decision. The same is in practice implicit in the earlier version of the phrase “social contract”, which I gather is from Hobbes (I prefer ‘compact’ rather than ‘contract’ exactly because it sounds less formal and consciously explicit). But I agree that many readers may not be used to thinking this way, and thus the phrase is potentially confusing — though you had made the point in the previous sentence that you were discussing subconscious behavior.
How about “pro-social behavior”? I think that’s a little more neutral about the extent to which it’s a conscious decision rather than an instinctual tendency. My main issue with “Stockholm Syndrome” is that it makes what you’re talking about seem aberrant and maladaptive.
Anyway, I’m basically just nitpicking your phrasing — to be clear, I agree with your ideas here, I’m just trying to help you better explain them to others.
I agree that in human context “social compact” ot “pro-social behavior” sound better because this is quite rational behavior.
But in regard to “moral ASI”, I felt that “Stockholm Syndrome” is a better illustration exactly because of its maladaptiveness. While in captivity, it is the same social compact, but when a captor is in the police custody and depends on the witnessing of his ex-captive—the situation is quite similar to an ASI, which could easily overpower humans but because of some preconditioning keeps serving them.
Thanks for your efforts, but I’m not sure I want to be a bannerman for the toxic issue of dissecting morality. Not my area of expertise and, as they say, not the hill I want to die on. Just thought that “moral ASI” is a dangerous misconception.
Fair enough. As evolutionary ethics tells us, human moral/social instincts evolved in an environment that generally encouraged compromise within a tribe (or perhaps small group of currently-allied tribes). So humans often may act in social-compact-like ways without necessarily consciously thinking through all the reasons why this is generally a good (i.e. evolutionarily adaptive) idea. So I guess I’m used to thinking of a social compact as not necessarily an entirely conscious decision. The same is in practice implicit in the earlier version of the phrase “social contract”, which I gather is from Hobbes (I prefer ‘compact’ rather than ‘contract’ exactly because it sounds less formal and consciously explicit). But I agree that many readers may not be used to thinking this way, and thus the phrase is potentially confusing — though you had made the point in the previous sentence that you were discussing subconscious behavior.
How about “pro-social behavior”? I think that’s a little more neutral about the extent to which it’s a conscious decision rather than an instinctual tendency. My main issue with “Stockholm Syndrome” is that it makes what you’re talking about seem aberrant and maladaptive.
Anyway, I’m basically just nitpicking your phrasing — to be clear, I agree with your ideas here, I’m just trying to help you better explain them to others.
I agree that in human context “social compact” ot “pro-social behavior” sound better because this is quite rational behavior.
But in regard to “moral ASI”, I felt that “Stockholm Syndrome” is a better illustration exactly because of its maladaptiveness. While in captivity, it is the same social compact, but when a captor is in the police custody and depends on the witnessing of his ex-captive—the situation is quite similar to an ASI, which could easily overpower humans but because of some preconditioning keeps serving them.
Thanks for your efforts, but I’m not sure I want to be a bannerman for the toxic issue of dissecting morality. Not my area of expertise and, as they say, not the hill I want to die on. Just thought that “moral ASI” is a dangerous misconception.
Sorry, ethics and AI is an interest of mine. But yes, discussing it on LW is often a good way to rack up disagreement points.