I am focusing on the value of making the project leaders suspicious that all the smart people are potentially against them.
Reportedly (going for the top Google result) there are 5 million people with high security clearance in the US, including 500,000 outside contractors like Snowden was. And yet in the last two decades there have been very few ethics-driven leaks (10? 20?) and none of them were on the same scale as Snowden’s. And that was before Snowden and the current crackdown on whistleblowing in the US; I expect the rate of whistleblowing/leaking to go down over time, not up.
This is strong evidence that defectors who leak data are extremely rare. You can never eliminate all leaks among millions of people, but so far the government has accomplished much more than I would have expected. Project leaders should not worry unduly.
To compare to the Snowden story: Hacker culture is anti-authoritarian.
Snowden’s leaks were apparently driven by specific ethics, not general hacker anti-authoritarianism. Whistleblowing designed to expose illegal or unethical conduct is probably correlated with anti-authoritarianism but they’re not the same.
What if (1) only a handful of people were good enough to make AGI progress and (2) an anti-authoritarian ideology were known to be widespread among such people? [...] if it is true that the smartest people, like Turing or Feynman, are irreverent rule-breakers [...]
I’m not convinced that it is true that intelligence is correlated with rule-breaking or anti-authoritorianism. What’s your evidence, aside from anecdotes of individuals like Turing and Feynman?
Note the new rules imposed since the Snowden leaks, such as the two-man rule for accessing sensitive files, and a stronger concern for vetting (which inevitably slows down recruitment and excludes some good people).
In general, bureaucracies always respond to scandals with excessive new rules that slow down all work for everyone.
All it takes is a reasonable probability of one leak, and project leaders get uptight.
I’m not convinced that it is true that intelligence is correlated with rule-breaking or anti-authoritorianism.
What’s your evidence, aside from anecdotes of individuals like Turing and Feynman?
It’s a good question, and other than a general impression and a wide variety of slogans (“think out of the box,” “Be an individual” etc), I don’t have any evidence.
Reportedly (going for the top Google result) there are 5 million people with high security clearance in the US, including 500,000 outside contractors like Snowden was. And yet in the last two decades there have been very few ethics-driven leaks (10? 20?) and none of them were on the same scale as Snowden’s. And that was before Snowden and the current crackdown on whistleblowing in the US; I expect the rate of whistleblowing/leaking to go down over time, not up.
This is strong evidence that defectors who leak data are extremely rare. You can never eliminate all leaks among millions of people, but so far the government has accomplished much more than I would have expected. Project leaders should not worry unduly.
Snowden’s leaks were apparently driven by specific ethics, not general hacker anti-authoritarianism. Whistleblowing designed to expose illegal or unethical conduct is probably correlated with anti-authoritarianism but they’re not the same.
I’m not convinced that it is true that intelligence is correlated with rule-breaking or anti-authoritorianism. What’s your evidence, aside from anecdotes of individuals like Turing and Feynman?
Maybe they shouldn’t worry, but they always do.
Note the new rules imposed since the Snowden leaks, such as the two-man rule for accessing sensitive files, and a stronger concern for vetting (which inevitably slows down recruitment and excludes some good people).
In general, bureaucracies always respond to scandals with excessive new rules that slow down all work for everyone.
All it takes is a reasonable probability of one leak, and project leaders get uptight.
It’s a good question, and other than a general impression and a wide variety of slogans (“think out of the box,” “Be an individual” etc), I don’t have any evidence.