“Security” for me has the connotation of being explicitely in relation to malicious adversaries, while “safety” carries the connotation of heavy industrial machinery that must be made as accident-free as possible. As an example, “boat security” would be preventing people from stealing the boat, while “boat safety” would be measures intended to prevent people from falling off the boat. AI alignment (which I consider a very good term!) strikes me to be more about the latter than the former (if you have a superintelligent adversary, it might already be too late!).
“Security” for me has the connotation of being explicitely in relation to malicious adversaries
But, like, in my view the main issue with advanced AI capabilities is that they come with an adversary built-in. This does make it fundamentally different from protecting against an external adversary, but I think the change is smaller than, say, the change from “preventing people from falling off the boat.” Like, the issue isn’t that the boat deck is going to be wet, or the sea will be stormy, or stuff like that; the issue is that the boat is thinking!
“Security” for me has the connotation of being explicitely in relation to malicious adversaries, while “safety” carries the connotation of heavy industrial machinery that must be made as accident-free as possible. As an example, “boat security” would be preventing people from stealing the boat, while “boat safety” would be measures intended to prevent people from falling off the boat. AI alignment (which I consider a very good term!) strikes me to be more about the latter than the former (if you have a superintelligent adversary, it might already be too late!).
But, like, in my view the main issue with advanced AI capabilities is that they come with an adversary built-in. This does make it fundamentally different from protecting against an external adversary, but I think the change is smaller than, say, the change from “preventing people from falling off the boat.” Like, the issue isn’t that the boat deck is going to be wet, or the sea will be stormy, or stuff like that; the issue is that the boat is thinking!