I don’t think either of these possibilities are really justified. We don’t necessarily know what capabilities are required to be an existential threat, and probably don’t even have a suitable taxonomy for classifying them that maps to real-world risk. What looks to us like conjunctional requirements may be more disjunctional than we think, or vice versa.
“Jagged” capabilities relative to human are bad if the capability requirements are more disjunctional than we think, since we’ll be lulled by low assessments in some areas that we think of as critical but actually aren’t.
They’re good if high risk requires more conjunctional capabilities than we think, especially if the AIs are jaggedly bad in actually critical areas that we don’t even know that we should be measuring.
I don’t think either of these possibilities are really justified. We don’t necessarily know what capabilities are required to be an existential threat, and probably don’t even have a suitable taxonomy for classifying them that maps to real-world risk. What looks to us like conjunctional requirements may be more disjunctional than we think, or vice versa.
“Jagged” capabilities relative to human are bad if the capability requirements are more disjunctional than we think, since we’ll be lulled by low assessments in some areas that we think of as critical but actually aren’t.
They’re good if high risk requires more conjunctional capabilities than we think, especially if the AIs are jaggedly bad in actually critical areas that we don’t even know that we should be measuring.