Come on, most every business tracks revenue in great detail. If customers were getting unhappy with the firm’s services and rapidly switching en mass, the firm would quickly become very aware, and looking into the problem in great detail.
RobinHanson
Karma: 4,461
You complain that my estimating rates from historical trends is arbitrary, but you offer no other basis for estimating such rates. You only appeal to uncertainty. But there are several other assumptions required for this doomsday scenario. If all you have is logical possibility to argue for piling on several a priori unlikely assumptions, it gets hard to take that seriously.
You keep invoking the scenario of a single dominant AI that is extremely intelligent. But that only happens AFTER a single AI fooms to be much better than all other AIs. You can’t invoke its super intelligence to explain why its owners fail to notice and control its early growth.
I comment on this paper here: https://www.overcomingbias.com/2022/07/cooks-critique-of-our-earliness-argument.html
That’s an exponential with mean 0.7, or mean 1⁄0.7?
“My prior on is distributed ”
I don’t understand this notation. It reads to me like “103+ 5 Gy”; how is that a distribution?
It seems the key feature of this remaining story is the “coalition of AIs” part. I can believe that AIs would get powerful, what I’m skeptical about is the claim that they naturally form a coalition of them against us. Which is also what I object to in your prior comments. Horses are terrible at coordination compared to humans, and humans weren’t built by horses and integrated into a horse society, with each human originally in the service of a particular horse.
Its not enough that AI might appear in a few decades, you also need something useful you can do about it now, compared to investing your money to have more to spend later when concrete problems appear.
I just read through your “what 2026 looks like” post, but didn’t see how it is a problematic scenario. Why should we want to work ahead of time to prepare for that scenario?
In our simulations, we find it overwhelmingly likely that any such spherical volume of an alien civ would be much larger than the full moon in the sky. So no need to study distant galaxies in fine detail; look for huge spheres in the sky.
“or more likely we are an early civilization in the universe (according to Robin Hanson’s “Grabby Aliens” model) so, 2) quite possibly there are no grabby aliens populating the universe with S-Risks yet”
But our model implies that there are in fact many aliens out there right now. Just not in our backward light cone.
Aw, I still don’t know which face goes with the TGGP name.
Wow, it seems that EVERYONE here has this counter argument “You say humans look weird according to this calculation, but here are other ways we are weird that you don’t explain.” But there is NO WAY to explain all ways we are weird, because we are in fact weird in some ways. For each way that we are weird, we should be looking for some other way to see the situation that makes us look less weird. But there is no guarantee of finding that; we just just actually be weird. https://www.overcomingbias.com/2021/07/why-are-we-weird.html
You have the date of the great filter paper wrong; it was 1998, not 1996.
Yes, a zoo hypothesis is much like a simulation hypothesis, and the data we use cannot exclude it. (Nor can they exclude a simulation hypothesis.) We choose to assume that grabby aliens change their volumes in some clearly visible way, exactly to exclude zoo hypotheses.
I’m arguing for simpler rules here overall.
Your point #1 misses the whole norm violation element. The reason it hurts if others are told about an affair is that others disapprove. That isn’t why loud music hurts.
Imagine there’s a law against tattoos, and I say “Yes some gang members wear them but so do many others. Maybe just outlaw gang tattoos?” You could then respond that I’m messing with edge cases, so we should just leave the rule alone.
You will allow harmful gossip, but not blackmail, because the first might be pursuing your “values”, but the second is seeking to harm. Yet the second can have many motives, and is mostly commonly to get money. And you are focused too much on motives, rather than on outcomes.
Seems to me I spent a big % of my post arguing against the rapid growth claim.