Maybe at this point we’re mostly arguing about what it means to live well? Material improvements matter to the extent that they make us happier and more satisfied. Perhaps I should have said 1% or something a bit smaller, but “0.0001%” and “all” seem more wrong in the other direction.
When I say the extreme “0.0001%” and “all” I’m mostly thinking about how I think near-optimal futures are nontrivially likely, how “AI going well” is what could enable those futures, and how near-optimal futures are not only somewhat better than the world today, but literally “extremely good” / much, much better.
That is, in these extremely good futures, I see us surviving to what Bostrom calls “technological maturity,” and I also see the population of humans and/or conscious minds we care about ballooning to astronomically large numbers as we engage in space colonization and put to use of much of the matter in the reachable universe (~400 billion stars in our galaxy, ~2 trillion galaxies in the reachable universe). The vast number of beings in these futures means that ensuring that 99.9999+% of them have better lives than all of us in 2026 is just a matter of being intentional about making sure that the minds we create in other galaxies are actually ~all awesome. I think technologically we’d easily have the capability and it’d just be a question of are we actually universally optimizing/trying for that or not. (In Letter from Utopia, I think Bostrom does a good job conveying that humans today have lives nowhere close to the best possible lives or experiences permitted by physics.) And again, in futures where AI goes well, I think there’s a significant chance (and it definitely “could” happen, to use the word you used when saying 10%/90%) that we would manage to actually optimize for such a near-optimal, extremely good future.
This seems like a conservative understatement to me. I think the bottom 0.0001% could be living far better than all humans do now.
Maybe at this point we’re mostly arguing about what it means to live well? Material improvements matter to the extent that they make us happier and more satisfied. Perhaps I should have said 1% or something a bit smaller, but “0.0001%” and “all” seem more wrong in the other direction.
When I say the extreme “0.0001%” and “all” I’m mostly thinking about how I think near-optimal futures are nontrivially likely, how “AI going well” is what could enable those futures, and how near-optimal futures are not only somewhat better than the world today, but literally “extremely good” / much, much better.
That is, in these extremely good futures, I see us surviving to what Bostrom calls “technological maturity,” and I also see the population of humans and/or conscious minds we care about ballooning to astronomically large numbers as we engage in space colonization and put to use of much of the matter in the reachable universe (~400 billion stars in our galaxy, ~2 trillion galaxies in the reachable universe). The vast number of beings in these futures means that ensuring that 99.9999+% of them have better lives than all of us in 2026 is just a matter of being intentional about making sure that the minds we create in other galaxies are actually ~all awesome. I think technologically we’d easily have the capability and it’d just be a question of are we actually universally optimizing/trying for that or not. (In Letter from Utopia, I think Bostrom does a good job conveying that humans today have lives nowhere close to the best possible lives or experiences permitted by physics.) And again, in futures where AI goes well, I think there’s a significant chance (and it definitely “could” happen, to use the word you used when saying 10%/90%) that we would manage to actually optimize for such a near-optimal, extremely good future.