I see this post is gathering downvotes (-3 so far) but no comments at all. It would be helpful if someone managed to put their reaction into words, and not just into a downvote.
Perhaps the “scenario” seems arbitrary or the purpose of the post is obscure. To some extent I was just musing aloud on the implications of a new fact. I knew intellectually that the NSA has its billion-dollar budgets and its thousands of PhD mathematicians, and the creation of AI in a secret military project is a standard fictional trope. But to hear about this specific facility concretized everything for me, and stirred my imagination.
My whimsy about a clique of singularitarian Mormon computer scientists may be somewhat arbitrary. But consider this: who is more likely to create the first AGI—the Singularity Institute, or the National Security Agency? The answer to that can hardly be in doubt. The NSA’s mission is to stay ahead of everyone else in matters like code-breaking and quantitative data analysis. They have to remain number one in theoretical computer science, and they have a budget of billions with which to accomplish that goal.
So if the future hinges on the value system of the first AI, then what goes on inside the NSA is far more important than what goes on at singinst.org. The Singularity Institute may have adopted a goal—design and create friendly AI—which, according to the Institute’s own philosophy, means that they would determine the future of the human race; and some of the controversy about the Institute, its methods, personalities, etc, is coming about because of this. But if you accept the philosophy, then the NSA is surely the number-one candidate to actually decide the fate of the world. Outsiders will not get to decide what happens; the most we can reasonably hope to do is to make correct observations that might be noticed and taken into account by the people on the inside who will, for better or worse, make the fateful decisions.
Of course it is theoretically possible that Google, IBM, the FSB, Japan’s biggest supercomputer… will instead be ground zero for the intelligence explosion. But I would think that the NSA is well ahead of all of them.
I almost downvoted this because when I clicked on it from my RSS reader, it appeared to have been posted on main LW instead of discussion (known bug). This might be the reason for a lot of mysterious downvoting, actually.
The NSA’s mission is to stay ahead of everyone else in matters like code-breaking and quantitative data analysis. They have to remain number one in theoretical computer science, and they have a budget of billions with which to accomplish that goal.
Almost inescapable conclusion, yes. The only thing I see against them to win is the fact, that they might be too slow. Someone might be a bit faster, what is a threat for them and what should accelerate them further.
I see this post is gathering downvotes (-3 so far) but no comments at all. It would be helpful if someone managed to put their reaction into words, and not just into a downvote.
Perhaps the “scenario” seems arbitrary or the purpose of the post is obscure. To some extent I was just musing aloud on the implications of a new fact. I knew intellectually that the NSA has its billion-dollar budgets and its thousands of PhD mathematicians, and the creation of AI in a secret military project is a standard fictional trope. But to hear about this specific facility concretized everything for me, and stirred my imagination.
My whimsy about a clique of singularitarian Mormon computer scientists may be somewhat arbitrary. But consider this: who is more likely to create the first AGI—the Singularity Institute, or the National Security Agency? The answer to that can hardly be in doubt. The NSA’s mission is to stay ahead of everyone else in matters like code-breaking and quantitative data analysis. They have to remain number one in theoretical computer science, and they have a budget of billions with which to accomplish that goal.
So if the future hinges on the value system of the first AI, then what goes on inside the NSA is far more important than what goes on at singinst.org. The Singularity Institute may have adopted a goal—design and create friendly AI—which, according to the Institute’s own philosophy, means that they would determine the future of the human race; and some of the controversy about the Institute, its methods, personalities, etc, is coming about because of this. But if you accept the philosophy, then the NSA is surely the number-one candidate to actually decide the fate of the world. Outsiders will not get to decide what happens; the most we can reasonably hope to do is to make correct observations that might be noticed and taken into account by the people on the inside who will, for better or worse, make the fateful decisions.
Of course it is theoretically possible that Google, IBM, the FSB, Japan’s biggest supercomputer… will instead be ground zero for the intelligence explosion. But I would think that the NSA is well ahead of all of them.
I almost downvoted this because when I clicked on it from my RSS reader, it appeared to have been posted on main LW instead of discussion (known bug). This might be the reason for a lot of mysterious downvoting, actually.
Almost inescapable conclusion, yes. The only thing I see against them to win is the fact, that they might be too slow. Someone might be a bit faster, what is a threat for them and what should accelerate them further.