I think you’re neglecting something when trying to determine the right group of experts for judging AGI risk. You consider experts on AI, but the AGI risk thesis is not just a belief on the behavior of AI, it is also a belief on our long-term future, and it is incompatible with many other beliefs intelligent people hold on our long-term future. Therefore I think you should also consider experts on humanity’s long-term future as relevant. As an analogy, if the question you want to answer is “Is the on Bible a correct account of the nature of God?”, you should not just ask experts on the Bible. Instead, you should broaden your question to “What religion is true?” and ask experts on religion in general.
A difficult aspect is that it’s not clear who is an expert on the long-term future, seeing as almost everybody is interested in the issue. Politicians, historians, economists, and philosophers are all candidates. Perhaps the best candidates are the super forecasters you mentioned.
As an analogy, if the question you want to answer is “Is the on Bible a correct account of the nature of God?”, you should not just ask experts on the Bible. Instead, you should broaden your question to “What religion is true?” and ask experts on religion in general.
One thing we notice when we look at differences among religious views in the population is that there is a substantial geographic component. People in Germany are a lot more likely than people in Japan to be Roman Catholics. People in Bangkok, Thailand are a lot more likely than people in Little Rock, Arkansas to be Buddhists.
There are also clusters of opinion in many other fields based on location (continental philosophy, Chicago school of economics, Washington consensus, etc.).
One thing we notice when we look at differences among religious views in the population is that there is a substantial geographic component. People in Germany are a lot more likely than people in Japan to be Roman Catholics. People in Bangkok, Thailand are a lot more likely than people in Little Rock, Arkansas to be Buddhists.
If I am not mistaken, Newtonian physics was accepted almost instantly in the anglosphere, while
Cartesian physics was dominant on the Continent for an extended period of time thereafter. Also, interpretations of probability (like Frequentism and Bayesianism) have often clustered by university (or even particular faculties within an individual university). So, physics and statistics seems to suffer from the same problem.
For religion, the difference seems to persist over millennia, even under reasonably close contact, except in specific circumstances of authority and conquest — whereas for science and technology, adoption spreads over years or decades whenever there’s close contact and comparison.
Religious conquests such as the Catholic conquest of South America don’t seem more common worldwide than persistent religious differences such as in India, where Christians have remained a 2% minority despite almost 2000 years of contact including trade, missionary work, and even occasional conquest (the British Raj).
With religion, whenever there aren’t authorities with the political power to expel heretics, syncretism and folk-religion develop — the blending of religious traditions, rather than the inexorable disproof and overturning of one by another.
This suggests that differences in religious practice do not reliably bring the socioeconomic and geopolitical advantages that differences in scientific and technological practice bring. If one of the major religions brought substantial advantages, we would not expect to see the persistence of religious differences over millennia that we do.
(IIRC, Newtonian physics spread to the Continent beginning with du Châtelet’s translation of the Principia into French, some sixty years after its original publication.)
First of all, it seems like the mechanisms for this happening are different for science. One rarely sees scientific ideas spread by conquest, or by the kind of social pressure brought to bear by religions. Nobody gets told by their mother to believe Newton’s Laws every week at college in the way they might be told to go to Mass.
Second, everyone is (according to religion) supposed to believe in and live by religion, whether they are educated or not. I’d have greater expectations that beliefs associated with education are geographically limited.
One rarely sees scientific ideas spread by conquest
Except when a more-scientific society kicks the hell out of the less-scientific society and takes it over (or, maybe, just takes over its land and women). Science is very helpful for building weapons.
Try being openly creationist at a major university, you’ll quickly discover the kind of social pressure science can bring to bear.
There were several people in my physics phd program who were openly creationist, and they were politely left alone. I don’t know of an environment more science-filled, and honestly I’ve never known a higher density of creationists.
The spread of scientific ideas from Europe was almost entirely by conquest.
Only in a very literal sense. Nobody said “let’s conquer everyone in order to spread the atomic of theory of matter” and once they conquered they didn’t execute people who didn’t believe in atoms or decide that people who don’t believe in atoms are not permitted to testify in court.
Try being openly creationist at a major university, you’ll quickly discover the kind of social pressure science can bring to bear.
That’s not the same kind of social pressure. I’m referring to one’s personal life, not one’s professional life.
The question dealt with beliefs. Brushing one’s teeth (or going to a doctor instead of a healer) is an action. Going to Mass is technically an action, but its primary effect is to instill a belief system in the child.
Warning: Reference class tennis below.
I think you’re neglecting something when trying to determine the right group of experts for judging AGI risk. You consider experts on AI, but the AGI risk thesis is not just a belief on the behavior of AI, it is also a belief on our long-term future, and it is incompatible with many other beliefs intelligent people hold on our long-term future. Therefore I think you should also consider experts on humanity’s long-term future as relevant. As an analogy, if the question you want to answer is “Is the on Bible a correct account of the nature of God?”, you should not just ask experts on the Bible. Instead, you should broaden your question to “What religion is true?” and ask experts on religion in general.
A difficult aspect is that it’s not clear who is an expert on the long-term future, seeing as almost everybody is interested in the issue. Politicians, historians, economists, and philosophers are all candidates. Perhaps the best candidates are the super forecasters you mentioned.
One thing we notice when we look at differences among religious views in the population is that there is a substantial geographic component. People in Germany are a lot more likely than people in Japan to be Roman Catholics. People in Bangkok, Thailand are a lot more likely than people in Little Rock, Arkansas to be Buddhists.
There are also clusters of opinion in many other fields based on location (continental philosophy, Chicago school of economics, Washington consensus, etc.).
If I am not mistaken, Newtonian physics was accepted almost instantly in the anglosphere, while Cartesian physics was dominant on the Continent for an extended period of time thereafter. Also, interpretations of probability (like Frequentism and Bayesianism) have often clustered by university (or even particular faculties within an individual university). So, physics and statistics seems to suffer from the same problem.
For religion, the difference seems to persist over millennia, even under reasonably close contact, except in specific circumstances of authority and conquest — whereas for science and technology, adoption spreads over years or decades whenever there’s close contact and comparison.
Religious conquests such as the Catholic conquest of South America don’t seem more common worldwide than persistent religious differences such as in India, where Christians have remained a 2% minority despite almost 2000 years of contact including trade, missionary work, and even occasional conquest (the British Raj).
With religion, whenever there aren’t authorities with the political power to expel heretics, syncretism and folk-religion develop — the blending of religious traditions, rather than the inexorable disproof and overturning of one by another.
This suggests that differences in religious practice do not reliably bring the socioeconomic and geopolitical advantages that differences in scientific and technological practice bring. If one of the major religions brought substantial advantages, we would not expect to see the persistence of religious differences over millennia that we do.
(IIRC, Newtonian physics spread to the Continent beginning with du Châtelet’s translation of the Principia into French, some sixty years after its original publication.)
First of all, it seems like the mechanisms for this happening are different for science. One rarely sees scientific ideas spread by conquest, or by the kind of social pressure brought to bear by religions. Nobody gets told by their mother to believe Newton’s Laws every week at college in the way they might be told to go to Mass.
Second, everyone is (according to religion) supposed to believe in and live by religion, whether they are educated or not. I’d have greater expectations that beliefs associated with education are geographically limited.
Except when a more-scientific society kicks the hell out of the less-scientific society and takes it over (or, maybe, just takes over its land and women). Science is very helpful for building weapons.
In those situations, though, the conquering force usually makes a point to avoid letting their core scientific insights spread, lest the tables turn.
Only if the conquered society survives. Often it doesn’t—look at what happened in the Americas.
That seems like a subset, not an exception. A conquered society which doesn’t survive is certainly not going to be adopting higher-tech weapons.
Well that didn’t stop Japan from getting western advisers to help with it’s modernization after Perry opened it.
Interestingly,
Science =/= technology. Profoundly anti scientific movements can make their point using tech. .bought on the open market.
The spread of scientific ideas from Europe was almost entirely by conquest.
Try being openly creationist at a major university, you’ll quickly discover the kind of social pressure science can bring to bear.
Although they might get told to go to the doctor and not the New Age healer.
There were several people in my physics phd program who were openly creationist, and they were politely left alone. I don’t know of an environment more science-filled, and honestly I’ve never known a higher density of creationists.
Only in a very literal sense. Nobody said “let’s conquer everyone in order to spread the atomic of theory of matter” and once they conquered they didn’t execute people who didn’t believe in atoms or decide that people who don’t believe in atoms are not permitted to testify in court.
That’s not the same kind of social pressure. I’m referring to one’s personal life, not one’s professional life.
Where do you draw the line, though ? Kids also get told to brush their teeth and to never play in traffic...
Ask Jiro, I’m not convinced a line exists.
The question dealt with beliefs. Brushing one’s teeth (or going to a doctor instead of a healer) is an action. Going to Mass is technically an action, but its primary effect is to instill a belief system in the child.