Extreme transparency increases the probability of government intervention at some late stage of the game (attempting to create a friendly AI constitutes attempting to create an extremely powerful weapon).
See what I did there? I changed the genre from sci-fi to political thriller.
So if the awareness that MIRI is working on an AGI in secret (or rather, without sharing details) happens to get to the government, and they consider it a powerful weapon as you say...then what? You know what they do to grandiose visionaries working on powerful weapons in their backyard who explicitly don’t want to share, and whose goals pretty clearly compromise their position, right?
Extreme transparency increases the probability of government intervention at some late stage of the game (attempting to create a friendly AI constitutes attempting to create an extremely powerful weapon).
See what I did there? I changed the genre from sci-fi to political thriller.
So if the awareness that MIRI is working on an AGI in secret (or rather, without sharing details) happens to get to the government, and they consider it a powerful weapon as you say...then what? You know what they do to grandiose visionaries working on powerful weapons in their backyard who explicitly don’t want to share, and whose goals pretty clearly compromise their position, right?