People have been destroying large parts of the world since the days of Genghis Khan. The Manhattan Project did indeed require the work of highly intelligent scientists, but they were only one component of the project and were successful not just because of their intelligence.
Yudkowsky is no doubt aware of these facts