Examples of things that might be in the current #6 (“known unknowns”) -- superhuman law (aside from the directly persuasion-heavy aspects), superhuman logistics (aside from robotics and long-term planning, think doing the stuff that data scientists already do but better—mostly this does not entail deeply superhuman planning and agency), mass surveillance, military tech not included in one of the above categories (eg missile detection).
It’s hard to say what fits in the current #8 (“unknown unknowns”) but mostly I’m thinking it’s a combination of things pretty much nobody has considered and people have considered but don’t take very seriously. It’s hard to say what counts prospectively but as an example, if sycophancy was like 1000x − 100,000x a bigger deal, sycophancy (from the perspective of 2021 where ~0.00% of ppl considered it a threat[1]) would count.
The relative ordering between the two implies that I think we’re more likely to be hit by known unknowns than unknown unknowns (ie greater probability x impact mass from the known unknowns). I haven’t thought deeply about this and am open to counterarguments, especially if you have a more crisp model! :)
Examples of things that might be in the current #6 (“known unknowns”) -- superhuman law (aside from the directly persuasion-heavy aspects), superhuman logistics (aside from robotics and long-term planning, think doing the stuff that data scientists already do but better—mostly this does not entail deeply superhuman planning and agency), mass surveillance, military tech not included in one of the above categories (eg missile detection).
It’s hard to say what fits in the current #8 (“unknown unknowns”) but mostly I’m thinking it’s a combination of things pretty much nobody has considered and people have considered but don’t take very seriously. It’s hard to say what counts prospectively but as an example, if sycophancy was like 1000x − 100,000x a bigger deal, sycophancy (from the perspective of 2021 where ~0.00% of ppl considered it a threat[1]) would count.
The relative ordering between the two implies that I think we’re more likely to be hit by known unknowns than unknown unknowns (ie greater probability x impact mass from the known unknowns). I haven’t thought deeply about this and am open to counterarguments, especially if you have a more crisp model! :)
I guess in a way it’s vaguely in the superpersuasion cluster but extremely noncenttal, especially from the perspective of someone in 2021.