I would like to agree with you but experience says otherwise. Tyrants have always been able to find enough professionals with dubious morals to further their plans.
In World War I, German Jewish scientists contributed to the German war effort. In World War II, refugee scientist contributed to the Allied war effort. Tyrants can shoot themselves in the foot quite effectively.
A few top physicists were left in Germany, including Heisenberg, but it was not enough to move the project forward, and it’s suspected that Heisenberg may have deliberately sabotaged the project.
But you have a point. So long as AGI is at the cutting edge, only a handful of top people can move it forward. As Moore’s Law of Mad Science has its effect, “ordinary” scientists will be enough.
(And to make it clear, I am not suggesting that the US government is tyrannical.)
if the project does not take safety into account, we want exactly this—so long as it doesn’t get close enough to success that failure involves paper-clipping the world.
I would like to agree with you but experience says otherwise. Tyrants have always been able to find enough professionals with dubious morals to further their plans.
In World War I, German Jewish scientists contributed to the German war effort. In World War II, refugee scientist contributed to the Allied war effort. Tyrants can shoot themselves in the foot quite effectively.
A few top physicists were left in Germany, including Heisenberg, but it was not enough to move the project forward, and it’s suspected that Heisenberg may have deliberately sabotaged the project.
But you have a point. So long as AGI is at the cutting edge, only a handful of top people can move it forward. As Moore’s Law of Mad Science has its effect, “ordinary” scientists will be enough.
(And to make it clear, I am not suggesting that the US government is tyrannical.)
There are plenty cases where government puts a bunch of incompetent people on a project and the project fails.
if the project does not take safety into account, we want exactly this—so long as it doesn’t get close enough to success that failure involves paper-clipping the world.