The possibility of an intelligence explosion seems to be an extraordinary belief.
Extraordinary compared to what? We already now that most people are insane, so that belief beeing not shared by almost everybody doesn’t make it unlikely a priori.
In some ways the intellgence explosion is a straightforward extrapolation of what we know at the moment, so I don’t think your critisism is valid here.
What evidence justified a prior strong enough as to be updated on a single paragraph, written in natural language, to the extent that you would afterwards devote your whole life to that possibility?
I think one could tell a reasonably competent physicist 50 years prior to Schrödinger how to derive quantum mechanics in one paragraph of natural language. Human language can contain lots of information, especially if speaker and listener already share a lot of concepts.
I’m not sure why you’ve written your comment, are you just using the opportunity to bring up this old topic again? I find myself irritated by this, even though I probably agree with you :)
For me there are a lot of things that sound completely sane but might be just bunk. Antimatter weapons, grey goo, the creation of planet eating black holes by particle accelerators, aliens or string theory. I don’t have the necessary education to discern those ideas from an intelligence explosion. They all sound like possibilities to me that might or might not be true. All I can do is to recognize their extraordinary status and subsequently demand the peer-review and assessment of those ideas by a larger and educated audience. Otherwise I run the risk of being swayed by the huge utility associated with those ideas, I run the risk of falling prey to a Pascalian mugger.
I think one could tell a reasonably competent physicist 50 years prior to Schrödinger how to derive quantum mechanics in one paragraph of natural language.
Is Luke Muehlhauser that competent when it comes to all the fields associated with artificial intelligence research?
I’m not sure why you’ve written your comment, are you just using the opportunity to bring up this old topic again?
It’s not old, it becomes more relevant each day. Since I first voiced skepticism about the topic they expanded to the point of having world-wide meetings. At least a few people playing devil’s advocate is a healthy exercise in my opinion :-)
Extraordinary compared to what? We already now that most people are insane, so that belief beeing not shared by almost everybody doesn’t make it unlikely a priori. In some ways the intellgence explosion is a straightforward extrapolation of what we know at the moment, so I don’t think your critisism is valid here.
I think one could tell a reasonably competent physicist 50 years prior to Schrödinger how to derive quantum mechanics in one paragraph of natural language. Human language can contain lots of information, especially if speaker and listener already share a lot of concepts.
I’m not sure why you’ve written your comment, are you just using the opportunity to bring up this old topic again? I find myself irritated by this, even though I probably agree with you :)
For me there are a lot of things that sound completely sane but might be just bunk. Antimatter weapons, grey goo, the creation of planet eating black holes by particle accelerators, aliens or string theory. I don’t have the necessary education to discern those ideas from an intelligence explosion. They all sound like possibilities to me that might or might not be true. All I can do is to recognize their extraordinary status and subsequently demand the peer-review and assessment of those ideas by a larger and educated audience. Otherwise I run the risk of being swayed by the huge utility associated with those ideas, I run the risk of falling prey to a Pascalian mugger.
Is Luke Muehlhauser that competent when it comes to all the fields associated with artificial intelligence research?
It’s not old, it becomes more relevant each day. Since I first voiced skepticism about the topic they expanded to the point of having world-wide meetings. At least a few people playing devil’s advocate is a healthy exercise in my opinion :-)
Are you playing devil’s advocate?
For what it’s worth, I consider these things to have very different levels and kinds of implausibility.