I’d personally put more like 90% on the claim (and it might depend a lot on what you mean by strong incentives).
Are you talking in-retrospect? If you currently assign only 90% to this claim, I would be very happy to take your money (I would say a reasonable definition that I think Joe would have accepted at the time that we would be dealing with more than $1 billion in annual expenditure towards this goal).
I… actually have trouble imagining any definition that isn’t already met, as people are clearly trying to do this right now. But like, still happy to take your money if you want to bet and ask some third-party to adjudicate.
I wasn’t talking in retrospect, but I meant something might larger than $1 billion by strong incentives and I really mean very specifically APS systems at the time when they are feasible to build.
The 10% would come from other approaches/architectures ending up being surprisingly better at the point when people could build APS systems. (E.g., you don’t need your AIs to have “the models they use in making plans represent with reasonable accuracy the causal upshot of gaining and maintaining different forms of power over humans and the real-world environment”.)
In further consideration I might be more like 95%, but 90% doesn’t seem crazy to me depending on the details of the operationalization.
I would put very high probabilities on “people would pay >$50 billion for a strong APS system right now”, so we presumably agree on that.
It’s really key to my perspective here that by the time that we can build APS systems, maybe something else which narrowly doesn’t meet this definition has come around and looks more competitive. There is something messy here because maybe there are strong incentives to build APS systems eventually, but this occurs substantially after full automation of the whole economy or similar by other systems. I was trying to exclude cases where substantially after human intellectual labor is totally obsolete, APS systems are strongly incentivized as this case is pretty different. (And other factors like “maybe we’ll have radical superbabies before we can build APS systems” factor in too, though again this is very sensitive to operationalization.)
Pretty surprising that the paper doesn’t give much indication to what counts as “strong incentives” (or at least not that I could find after searching for 2 mins).
Are you talking in-retrospect? If you currently assign only 90% to this claim, I would be very happy to take your money (I would say a reasonable definition that I think Joe would have accepted at the time that we would be dealing with more than $1 billion in annual expenditure towards this goal).
I… actually have trouble imagining any definition that isn’t already met, as people are clearly trying to do this right now. But like, still happy to take your money if you want to bet and ask some third-party to adjudicate.
I wasn’t talking in retrospect, but I meant something might larger than $1 billion by strong incentives and I really mean very specifically APS systems at the time when they are feasible to build.
The 10% would come from other approaches/architectures ending up being surprisingly better at the point when people could build APS systems. (E.g., you don’t need your AIs to have “the models they use in making plans represent with reasonable accuracy the causal upshot of gaining and maintaining different forms of power over humans and the real-world environment”.)
In further consideration I might be more like 95%, but 90% doesn’t seem crazy to me depending on the details of the operationalization.
I would put very high probabilities on “people would pay >$50 billion for a strong APS system right now”, so we presumably agree on that.
It’s really key to my perspective here that by the time that we can build APS systems, maybe something else which narrowly doesn’t meet this definition has come around and looks more competitive. There is something messy here because maybe there are strong incentives to build APS systems eventually, but this occurs substantially after full automation of the whole economy or similar by other systems. I was trying to exclude cases where substantially after human intellectual labor is totally obsolete, APS systems are strongly incentivized as this case is pretty different. (And other factors like “maybe we’ll have radical superbabies before we can build APS systems” factor in too, though again this is very sensitive to operationalization.)
Pretty surprising that the paper doesn’t give much indication to what counts as “strong incentives” (or at least not that I could find after searching for 2 mins).