They think a friendly-AGI-run society would have some use for money, conflict, etc. I’d say the onus is on them to explain why we would need those things in such a society.
Because of the laws of thermodynamics holding, basically. I do buy that a lot of stuff could switch over to non-money modes, but if we assume that the basic laws of physics fundamentally still hold true, then this can’t work, and this is one of those areas where you need to give hard evidence.
Much more generally, the Industrial Revolution is a good example, in that it really did improve the lives of humans massively, even with imperfect distribution of benefits, but it didn’t end conflict or money, and I’d argue there was a use for money (Although the Industrial Revolution did drastically reduce the benefits of war to non-ideological actors.)
They think a good “singularity” would not be particularly “weird” or sci-fi looking, which ignores the evidence of technological development throughout history. I think this is what the “The specific, real reality in front of you” sentence is about.
Interestingly enough, while I think this is true over the long-term, and potentially even over the short term, I think a major problem is LWers tend to underestimate how long things take to change, and in general have a bit of a bad habit of assuming everything changing at maximum speed.
A medieval peasant would very much disagree with that sentence, if they were suddenly thrust into a modern grocery store. I think they would say the physical reality around them changed to a pretty magical-seeming degree.
I agree that the medieval peasant would be very surprised at how much things changed, but they’d also detect a lot of continuity, and would have a lot of commonalities, especially on the human side of things.
This seems to be a combo of the absurdity heuristic and trying to “psychoanalyze your way to the truth”. Just because something sounds kind of like some elements of some religions, does not make it automatically false.
But it does decrease the credence, potentially substantially, and that could be important.
Now, my general view is that I do think there’s reason to believe that AI could be the greatest technology in history, but I agree with the OP that there’s a little magic often involved, and it’s a little bit of a red flag how much AI gets compared to gods.
And contra some people, I do think the psychoanalyze your way to the truth is more useful than people think, especially if you have good a priori reason to expect biases to drive the discussion, because they can allow you to detect red flags.
Because of the laws of thermodynamics holding, basically. I do buy that a lot of stuff could switch over to non-money modes, but if we assume that the basic laws of physics fundamentally still hold true, then this can’t work, and this is one of those areas where you need to give hard evidence.
Much more generally, the Industrial Revolution is a good example, in that it really did improve the lives of humans massively, even with imperfect distribution of benefits, but it didn’t end conflict or money, and I’d argue there was a use for money (Although the Industrial Revolution did drastically reduce the benefits of war to non-ideological actors.)
Interestingly enough, while I think this is true over the long-term, and potentially even over the short term, I think a major problem is LWers tend to underestimate how long things take to change, and in general have a bit of a bad habit of assuming everything changing at maximum speed.
I agree that the medieval peasant would be very surprised at how much things changed, but they’d also detect a lot of continuity, and would have a lot of commonalities, especially on the human side of things.
But it does decrease the credence, potentially substantially, and that could be important.
Now, my general view is that I do think there’s reason to believe that AI could be the greatest technology in history, but I agree with the OP that there’s a little magic often involved, and it’s a little bit of a red flag how much AI gets compared to gods.
And contra some people, I do think the psychoanalyze your way to the truth is more useful than people think, especially if you have good a priori reason to expect biases to drive the discussion, because they can allow you to detect red flags.