The detail that stuck with me is that camera controls worked where ‘fair-play committees’ didn’t. That maps onto something I keep seeing from a governance angle: the deterrent isn’t the rule, it’s whether a functioning mechanism sits behind it. The Metta case basically established that the rule existed but the enforcement didn’t, and once that was common knowledge, cheating became the default not because norms collapsed, but because the cost of enforcement (social backlash, appeals, organizer futility) exceeded the cost of looking away.
What I find genuinely unsettling in your framing is the combination: gradual disempowerment doesn’t need superhuman AI AND it doesn’t need broken norms. It just needs the enforcement mechanism to be slightly more expensive than the act it’s meant to deter. The Go players had the norm (‘AI cheating is dishonorable’) intact the whole time arguably the vilification made enforcement harder, not easier. So the disempowerment ran straight through a fully intact value system.
Two questions this raises for me. First: is the ‘illusion of control’ you describe load-bearing, or incidental? I.e. would removing it (somehow making users vividly aware they ‘haven’t got it’) actually change behavior, or would they disempower themselves anyway because the obstacle-avoidance gradient is doing the real work? Second, on the cultural-evolution rebuttal in Appendix A the pre move 60 vs post move 60 split is a sharp test and I’d want to know whether it replicates in other domains where people copy AI policy on the ‘opening’ and then go off-script. If it does, that’s a fairly general diagnostic for ’mimicry mistaken for skill.
The detail that stuck with me is that camera controls worked where ‘fair-play committees’ didn’t. That maps onto something I keep seeing from a governance angle: the deterrent isn’t the rule, it’s whether a functioning mechanism sits behind it. The Metta case basically established that the rule existed but the enforcement didn’t, and once that was common knowledge, cheating became the default not because norms collapsed, but because the cost of enforcement (social backlash, appeals, organizer futility) exceeded the cost of looking away.
What I find genuinely unsettling in your framing is the combination: gradual disempowerment doesn’t need superhuman AI AND it doesn’t need broken norms. It just needs the enforcement mechanism to be slightly more expensive than the act it’s meant to deter. The Go players had the norm (‘AI cheating is dishonorable’) intact the whole time arguably the vilification made enforcement harder, not easier. So the disempowerment ran straight through a fully intact value system.
Two questions this raises for me.
First: is the ‘illusion of control’ you describe load-bearing, or incidental? I.e. would removing it (somehow making users vividly aware they ‘haven’t got it’) actually change behavior, or would they disempower themselves anyway because the obstacle-avoidance gradient is doing the real work?
Second, on the cultural-evolution rebuttal in Appendix A the pre move 60 vs post move 60 split is a sharp test and I’d want to know whether it replicates in other domains where people copy AI policy on the ‘opening’ and then go off-script. If it does, that’s a fairly general diagnostic for ’mimicry mistaken for skill.