I agree. Limited resources drives the need to have abstractions. Communication probably simply provides another drive to abstract at even higher levels.
I would imagine for many animals evolution would have pre-programmed abstractions like “object” or “fruits” so they come pre-parsed.
Building devices is actually like setting up physical experiments. If they do what they’re supposed to do, you can increase your confidence in the mechanisms that explain how they work.
Many of the neuroscience discoveries happened via a mix of luck and detective work—which is what I took from the post as a way for progressing science of complex systems.
Yes, you’ve paraphrased the argument well.
While progress for any certain technology can certainly be exponential, the benefit or impact of it may not be exponential if we assume that the lowest hanging fruits (in terms of cost/benefit tradeoff) gets plucked first, and then it becomes harder and harder to derive economically relevant benefit even with tech progress.
What unbounded benefit relies on whether we can keep on discovering new types of general-purpose technologies, and not just an improvement in current technologies.
E.g. faster air travel depends on some new uncertain technology to be invented. We can only squeeze incremental benefits from further improving the current air travel technologies.
Also see Eroom’s law which suggests declining R&D productivity for drug discovery.
Whether we can keep on discovering such general-purpose technologies is pure speculation, so there’s some probability that we may actually not be in a weird cross-roads within the next 200 years.
This is interesting. I agree that the various cleanup technologies add to the economic growth.
My point was more suggestive in the sense of whether that poses a practical constraint to economic growth (in terms of say negative feedback loops) when the cost of cleanup outstrips the benefit derived (and hence putting an upper limit to the desire of the benefit).
Source?
>Rather, it can produce goodness and empirically in the case of human evolution it has created things that we would call goodness.
Hasn’t it produced cooperation and goodness amongst humans who’re of similar power? My intuition pump for AI risk is what humans have done to other beings sharing the planet and the story there is not encouraging.