I tend to think differently on this one.
Wherever I turn my head around in this world, I see lost causes everywhere. I see Goodhart’s law and Campbell’s law at loose everywhere. I see insane optimizers everywhere. Political parties that concentrate more on show, pomp and campaign funds than on actual issues. Corporates that seek money to the exclusion of actual creation of value. Governments that seek employment and GDP growth even when those are supported by artificial stimuli and not sustainable patterns of production and trade.
One might argue that none of these systems are actually as intelligent as a well educated human at any given moment in time. But that’s the point, isn’t it? You’re unable to stop sub-human optimizers, how are you going to curb a near human or a super human one?
For me, the scary idea is not so much of an idea as it is an extension of something that is already happening in this world.
I put in $500, really pinches in Indian rupees (Rs. 23,000+). Hoping for the best to happen next year with a successful book release and promising research to be done.