I think we should apply Doomsday Argument only to intelligent observers who think about anthropics, not to stones, animals or famers as I am not randomly selected from them. Such observers started to appear in 1970 (if we ignore Laplace in 1801 who analyzed a similar problem but not exactly) and thus we get 50 per cent of Doomsday in 2082.
This matches our expectations of doom from AI and other x-risks and does not provide much new information.
Heh, funny, but this has two main issues. First, it’s a pretty obviously artificial reference class. Second, it runs into the same issue I mention in my closing paragraph. I could easily imagine that people lose interest in these weird anthropic arguments in a few decades without that actually being a bad thing for humanity.
I think it is less artificial than most other classes and I would even call it “natural”—but also each reference class has its own end. I also agree tha DA not necessary predicts extinction—but just the end of the reference class which can be lose of interest.
But more interesting is that out approaches actually produced similar results: you got around 5-6 per cent of doom in the next century—and I got just over 50. This is not a big difference when we speak about very vague predictions: Despite very large and very small reference classes, we get rather similar predictions and it is an argument for DA.
I feel like the most natural reference class would be conscious observers weighted by level of consciousness. I.e. I would regard any particular quale that I experience as being one randomly chosen quale from the space of all qualia that have ever existed or will ever exist in any observer in the multiverse.
If early hominids were of a similar level of consciousness to modern humans then that gives us a later expected doomsday. Conversely if the level of consciousness of the average human increased over the 20th century when the Flynn effect was underway then that gives us an earlier expected doomsday. Or maybe one can imagine weird scenarios where the correlation between intelligence and level of consciousness is very weak, but the correlation between number of autistic traits and level of consciousness is very strong and taking into account the rise in autism prevalence over the past several decades massively skews our estimated doomsday date.
There is SSSSA – super strong sampling assumption, which suggests to weight minds proportionally to their intelligence and it is similar to your proposal. https://arxiv.org/abs/1705.03078 It has its own Doomsday argument—that superintelligent minds are impossible.
However, my argument doesn’t depend on that and it claims that I should exclude from my reference class all minds which are definitely not in my epistemic situation. For example, Sleeping Beauty always knows all her experiment details. But if she forgets all of them on every Tuesday and knows that on Monday, she can reason: “as I now know about anthropics, I am not on Tuesday” and this change the whole experiment. (However, this is not related to some random elements of her experience, but only to the necessary ones, and here is the difference with Full Non-indexical Conditioning and Beauty-Technicolor.)
In other words, I treat all my non-necessary parts as randomly selected, and this includes my birthday date in some extent.
Regarding the linked paper: Let’s take SSSSA but weight by level of consciousness rather than by level of intelligence. Then anthropic reasoning rules out super-consciousness but does not rule out super-intelligence. In particular, anthropic reasoning is enough to rule out scenarios where there are galaxies full of enormous numbers of conscious AIs or real or uploaded humans. However it’s not enough to rule out scenarios where an unconscious or minimally conscious AI becomes smart enough to outperform humans in every domain. In fact, Integrated information Theory suggests that current AIs are not particularly conscious and we’re not far off from AGI.
Yes, agree about this consciousness version of Doomsday argument. But what is limited: number of different qualia? Number of qualia that can be expirienced simultaneously? Clarity?
I think we should apply Doomsday Argument only to intelligent observers who think about anthropics, not to stones, animals or famers as I am not randomly selected from them. Such observers started to appear in 1970 (if we ignore Laplace in 1801 who analyzed a similar problem but not exactly) and thus we get 50 per cent of Doomsday in 2082.
This matches our expectations of doom from AI and other x-risks and does not provide much new information.
Heh, funny, but this has two main issues. First, it’s a pretty obviously artificial reference class. Second, it runs into the same issue I mention in my closing paragraph. I could easily imagine that people lose interest in these weird anthropic arguments in a few decades without that actually being a bad thing for humanity.
I think it is less artificial than most other classes and I would even call it “natural”—but also each reference class has its own end. I also agree tha DA not necessary predicts extinction—but just the end of the reference class which can be lose of interest.
But more interesting is that out approaches actually produced similar results: you got around 5-6 per cent of doom in the next century—and I got just over 50. This is not a big difference when we speak about very vague predictions: Despite very large and very small reference classes, we get rather similar predictions and it is an argument for DA.
I feel like the most natural reference class would be conscious observers weighted by level of consciousness. I.e. I would regard any particular quale that I experience as being one randomly chosen quale from the space of all qualia that have ever existed or will ever exist in any observer in the multiverse.
If early hominids were of a similar level of consciousness to modern humans then that gives us a later expected doomsday. Conversely if the level of consciousness of the average human increased over the 20th century when the Flynn effect was underway then that gives us an earlier expected doomsday. Or maybe one can imagine weird scenarios where the correlation between intelligence and level of consciousness is very weak, but the correlation between number of autistic traits and level of consciousness is very strong and taking into account the rise in autism prevalence over the past several decades massively skews our estimated doomsday date.
There is SSSSA – super strong sampling assumption, which suggests to weight minds proportionally to their intelligence and it is similar to your proposal. https://arxiv.org/abs/1705.03078 It has its own Doomsday argument—that superintelligent minds are impossible.
However, my argument doesn’t depend on that and it claims that I should exclude from my reference class all minds which are definitely not in my epistemic situation. For example, Sleeping Beauty always knows all her experiment details. But if she forgets all of them on every Tuesday and knows that on Monday, she can reason: “as I now know about anthropics, I am not on Tuesday” and this change the whole experiment. (However, this is not related to some random elements of her experience, but only to the necessary ones, and here is the difference with Full Non-indexical Conditioning and Beauty-Technicolor.)
In other words, I treat all my non-necessary parts as randomly selected, and this includes my birthday date in some extent.
Ok, sorry you’re right about that.
Regarding the linked paper: Let’s take SSSSA but weight by level of consciousness rather than by level of intelligence. Then anthropic reasoning rules out super-consciousness but does not rule out super-intelligence. In particular, anthropic reasoning is enough to rule out scenarios where there are galaxies full of enormous numbers of conscious AIs or real or uploaded humans. However it’s not enough to rule out scenarios where an unconscious or minimally conscious AI becomes smart enough to outperform humans in every domain. In fact, Integrated information Theory suggests that current AIs are not particularly conscious and we’re not far off from AGI.
Yes, agree about this consciousness version of Doomsday argument. But what is limited: number of different qualia? Number of qualia that can be expirienced simultaneously? Clarity?
Weight observer moments by the number of bits of integrated information?