You say ‘That’s not how it works.’ But I think that IS how it works!
If progress were only ever made by people as smart as E.T. Jaynes, humanity would never have gotten anywhere. Even with fat tails, intelligence is still roughly normally distributed, and there just aren’t that many 6 sigma events. The vast majority of scientific progress is incremental, notwithstanding that it’s only the revolutionary achievements that are salient.
The real question is, do you want Friendly A.I. to be achieved? Or do you just want friendly A.I. to be achieved by YOU? There’s no shame in the latter one, but the preclusion of the latter speaks little about progress towards the former (which I happen to think this blog is immensely valuable towards).
You say ‘That’s not how it works.’ But I think that IS how it works!
If progress were only ever made by people as smart as E.T. Jaynes, humanity would never have gotten anywhere. Even with fat tails, intelligence is still roughly normally distributed, and there just aren’t that many 6 sigma events. The vast majority of scientific progress is incremental, notwithstanding that it’s only the revolutionary achievements that are salient.
The real question is, do you want Friendly A.I. to be achieved? Or do you just want friendly A.I. to be achieved by YOU? There’s no shame in the latter one, but the preclusion of the latter speaks little about progress towards the former (which I happen to think this blog is immensely valuable towards).