I would like to bring forward from 2017 the paper from Ministry of Foreign Affairs of Finland, done with cooperation of Global Priorities Project and Future of Humanity Institute, University of Oxford
1.1.4 Artificial intelligence
Currently, artificial intelligence can outperform humans in a number of narrow domains, such as playing chess and searching data. As artificial intelligence researchers continue to make progress, though, these domains are highly likely to grow in number and breadth over time.
Many experts now believe there is a significant chance that a machine superintelligence – a system that can outperform humans at all relevant intelligence tasks – will be developed within the next century. In a 2014 survey of artificial intelligence experts, the median expert estimated that there is a 50% chance of human-level artificial intelligence by 2040, and that once human-level artificial intelligence is achieved, there is a 75% chance of superintelligence in the following 30 years.44 Although small sample size, selection bias, and the unreliability of subjective opinions mean that these estimates warrant scepticism, they nevertheless suggest that the possibility of superintelligence ought to be taken seriously.
If a superintelligence comes to exist, it will plausibly usher in economic, social, and political changes of a magnitude significantly beyond those wrought by the Industrial Revolution. While it could certainly offer many benefits, such as increased economic productivity and solutions to various technical problems, superintelligence could also be a factor in increasing existential risk.
Firstly, it could exacerbate other existential risks by destabilising political equilibria or by enabling the creation and deployment of other dangerous technologies. Secondly, it could cause grave harm through unintended consequences: the technology could be so opaque and powerful as to make it hard to ensure that it behaves in a way conducive to human good. There are a number of difficult technical problems related to the design of accident-free artificial-intelligence systems that have only recently been recognised.45 If superintelligence comes to exist before these problems are solved then it could itself constitute an existential risk.46
Given 2017+30 = 2047, and the passage mentions 50% for AGI before 2040, that would seem to match to “demonstrated past pragmatic power to control the disposition of funding and public promotion of ideas, contrary to “AGI median in 30 years or longer” and “utter ruin at 10% or lower”, before the ChatGPT moment.” per my understanding.
I would like to bring forward from 2017 the paper from Ministry of Foreign Affairs of Finland, done with cooperation of Global Priorities Project and Future of Humanity Institute, University of Oxford
Existential Risk Diplomacy and Governance, 1.1.4 Artificial Intelligence, page 9
Given 2017+30 = 2047, and the passage mentions 50% for AGI before 2040, that would seem to match to “demonstrated past pragmatic power to control the disposition of funding and public promotion of ideas, contrary to “AGI median in 30 years or longer” and “utter ruin at 10% or lower”, before the ChatGPT moment.” per my understanding.