Basically nobody actually wants the world to end, so if we do that to ourselves, it will be because somewhere along the way we weren’t good enough at navigating collective action problems, institutional steering, and general epistemics
… or because we didn’t understand important stuff well enough in time (for example: if it is the case that by default, the first AI that could prove P≠NP would eat the Sun, we would want to firmly understand this ahead of time), or because we weren’t good enough at thinking (for example, people could just be lacking in iq, or have never developed an adequate sense of what it is even like to understand something, or be intellectually careless), or because we weren’t fast enough at disseminating or [listening to] the best individual understanding in critical cases, or because we didn’t value the right kinds of philosophical and scientific work enough, or because we largely-ethically-confusedly thought some action would not end the world despite grasping some key factual broad strokes of what would happen after, or because we didn’t realize we should be more careful, or maybe because generally understanding what will happen when you set some process in motion is just extremely cursed.[1] I guess one could consider each of these to be under failures in general epistemics… but I feel like just saying “general epistemics” is not giving understanding its proper due here.
Sure, I’m definitely eliding a bunch of stuff here. Actually one of the things I’m pretty confused about is how to carve up the space, and what the natural category for all this is: epistemics feels like a big stretch. But there clearly is some defined thing that’s narrower than ‘get better at literally everything’.
… or because we didn’t understand important stuff well enough in time (for example: if it is the case that by default, the first AI that could prove P≠NP would eat the Sun, we would want to firmly understand this ahead of time), or because we weren’t good enough at thinking (for example, people could just be lacking in iq, or have never developed an adequate sense of what it is even like to understand something, or be intellectually careless), or because we weren’t fast enough at disseminating or [listening to] the best individual understanding in critical cases, or because we didn’t value the right kinds of philosophical and scientific work enough, or because we largely-ethically-confusedly thought some action would not end the world despite grasping some key factual broad strokes of what would happen after, or because we didn’t realize we should be more careful, or maybe because generally understanding what will happen when you set some process in motion is just extremely cursed.[1] I guess one could consider each of these to be under failures in general epistemics… but I feel like just saying “general epistemics” is not giving understanding its proper due here.
Many of these are related and overlapping.
Sure, I’m definitely eliding a bunch of stuff here. Actually one of the things I’m pretty confused about is how to carve up the space, and what the natural category for all this is: epistemics feels like a big stretch. But there clearly is some defined thing that’s narrower than ‘get better at literally everything’.