Just stumbled upon this post by Nate where he describes how he… hacked his System 1 to ignore any Knightian uncertainty and unknown unknowns? Which is, like… the textbook way to make sure that you’re wildly uncalibrated a few years down the line, and in fact precisely what has happened. Man.
I have invoked Willful Inconsistency on only two occasions, and they were similar in nature. Only one instance of Willful Inconsistency is currently active, and it works like this:
I have completely and totally convinced my intuitions that unfriendly AI is a problem. A big problem. System 1 operates under the assumption that UFAI will come to pass in the next twenty years with very high probability.
You can imagine how this is somewhat motivating.
On the conscious level, within System 2, I’m much less certain. I solidly believe that UFAI is a big problem, and that it’s the problem that I should be focusing my efforts on. However, my error bars are far wider, my timespan is quite broad. I acknowledge a decent probability of soft takeoff. I assign moderate probabilities to a number of other existential threats. I think there are a large number of unknown unknowns, and there’s a non-zero chance that the status quo continues until I die (and that I can’t later be brought back). All this I know.
But, right now, as I type this, my intuition is screaming at me that the above is all wrong, that my error bars are narrow, and that I don’t actually expect the status quo to continue for even thirty years.
I have met people within effective altruism who seem to be trying to do scary, dark things to their beliefs/motivations, which feels in the same category, like trying to convince themselves they don’t care about anything besides maximising impact or reducing x-risk. The latter, in at least one case, by thinking lots about dying due to AI to start caring about it more, which can’t be good for thinking clearly in the way they described it.
Just stumbled upon this post by Nate where he describes how he… hacked his System 1 to ignore any Knightian uncertainty and unknown unknowns? Which is, like… the textbook way to make sure that you’re wildly uncalibrated a few years down the line, and in fact precisely what has happened. Man.
Wow, the quoted text feels scary to read.
I have met people within effective altruism who seem to be trying to do scary, dark things to their beliefs/motivations, which feels in the same category, like trying to convince themselves they don’t care about anything besides maximising impact or reducing x-risk. The latter, in at least one case, by thinking lots about dying due to AI to start caring about it more, which can’t be good for thinking clearly in the way they described it.