“AI Safety” which often in practice means “self driving cars”
This may have been true four years ago, but ML researchers at leading labs rarely directly work on self-driving cars (e.g., research on sensor fusion). AV is has not been hot in quite a while. Fortunately now that AGI-like chatbots are popular, we’re moving out of the realm of talking about making very narrow systems safer. The association with AV was not that bad since it was about getting many nines of reliability/extreme reliability, which was a useful subgoal. Unfortunately the world has not been able to make a DL model completely reliable in any specific domain (even MNIST).
Of course, they weren’t talking about x-risks, but neither are industry researchers using the word “alignment” today to mean they’re fine-tuning a model to be more knowledgable or making models better satisfy capabilities wants (sometimes dressed up as “human values”).
If you want a word that reliably denotes catastrophic risks that is also mainstream, you’ll need to make catastrophic risk ideas mainstream. Expect it to be watered down for some time, or expect it not to go mainstream.
Unfortunately, I think even “catastrophic risk” has a high potential to be watered down and be applied to situations where dozens as opposed to millions/billions die. Even existential risk has this potential, actually, but I think it’s a safer bet.
This may have been true four years ago, but ML researchers at leading labs rarely directly work on self-driving cars (e.g., research on sensor fusion). AV is has not been hot in quite a while. Fortunately now that AGI-like chatbots are popular, we’re moving out of the realm of talking about making very narrow systems safer. The association with AV was not that bad since it was about getting many nines of reliability/extreme reliability, which was a useful subgoal. Unfortunately the world has not been able to make a DL model completely reliable in any specific domain (even MNIST).
Of course, they weren’t talking about x-risks, but neither are industry researchers using the word “alignment” today to mean they’re fine-tuning a model to be more knowledgable or making models better satisfy capabilities wants (sometimes dressed up as “human values”).
If you want a word that reliably denotes catastrophic risks that is also mainstream, you’ll need to make catastrophic risk ideas mainstream. Expect it to be watered down for some time, or expect it not to go mainstream.
Unfortunately, I think even “catastrophic risk” has a high potential to be watered down and be applied to situations where dozens as opposed to millions/billions die. Even existential risk has this potential, actually, but I think it’s a safer bet.