Yes you hit the nail on the head understanding my point, thank you. I also think this is what Yann is saying, to go out on a limb: He’s doing AI-safety simultaneously, he considers alignment AS safety.
I guess, maybe, I can see how the 2nd take could be true..but I also can’t think of a practical example, which is my sticking point. Of course, a bomb which can blow-up the moon is partly “capable”, and there is partial-progress to report—but only if we judge it based on limited factors, and exclude certain essential ones (e.g. navigation). I posit we will never avoid judging our real inventions based on what I’d consider essential output:
“Will it not kill us == Does it work?”
It’s a theory, but: I think AI-safety ppl may lose the argument right away, and can sadly be an afterthought (that’s what I’m told, by them), because they are allowing others to define “intelligence/capability” to be free from normal human concerns about our own safety...like I said before, others can go their merry-way making stuff more powerful, calling it “progress”, calling it higher-IQ...but I don’t see how that should earn Capability.
Ah, I see. I thought we were having a sticking point on definitions, but it seems that the definition is part of the point.
So, if I have this right, what you’re saying is:
Currently, the AI community defines capability and safety as two different things. This is very bad. Firstly, because it’s wrong—an unsafe system cannot reasonably be thought of as being capable of achieving anything more complex than predicting cat pictures. Secondly, because it leads to bad outcomes when this paradigm is adopted by AI researchers. Who doesn’t want to make a more capable system? Who wants to slow that down for “safety”? That shit’s boring! What would be better is if the AI community considered safety to be a core metric of capability, just as important as “Is this AI powerful enough to perform the task we want?”.
Glad to help! And hey, clarifying our ideas is half of what discussion is for!
I’d love to see a top-level post on ideas for making this happen, since I think you’re right, even though safety in current AI systems is very different from the problems we would face with AGI-level systems.
but I’m prob going to stay in the “dumb questions” area and not comment :)
ie. “the feeling I have when someone tries to teach me that human-safety is orthogonal to AI-Capability—in a real implementation, they’d be correlated in some way”
Yes you hit the nail on the head understanding my point, thank you. I also think this is what Yann is saying, to go out on a limb: He’s doing AI-safety simultaneously, he considers alignment AS safety.
I guess, maybe, I can see how the 2nd take could be true..but I also can’t think of a practical example, which is my sticking point. Of course, a bomb which can blow-up the moon is partly “capable”, and there is partial-progress to report—but only if we judge it based on limited factors, and exclude certain essential ones (e.g. navigation). I posit we will never avoid judging our real inventions based on what I’d consider essential output:
“Will it not kill us == Does it work?”
It’s a theory, but: I think AI-safety ppl may lose the argument right away, and can sadly be an afterthought (that’s what I’m told, by them), because they are allowing others to define “intelligence/capability” to be free from normal human concerns about our own safety...like I said before, others can go their merry-way making stuff more powerful, calling it “progress”, calling it higher-IQ...but I don’t see how that should earn Capability.
Ah, I see. I thought we were having a sticking point on definitions, but it seems that the definition is part of the point.
So, if I have this right, what you’re saying is:
Currently, the AI community defines capability and safety as two different things. This is very bad. Firstly, because it’s wrong—an unsafe system cannot reasonably be thought of as being capable of achieving anything more complex than predicting cat pictures. Secondly, because it leads to bad outcomes when this paradigm is adopted by AI researchers. Who doesn’t want to make a more capable system? Who wants to slow that down for “safety”? That shit’s boring! What would be better is if the AI community considered safety to be a core metric of capability, just as important as “Is this AI powerful enough to perform the task we want?”.
YES.
You are a gentleman and a scholar for taking the time on this. I wish I could’ve explained it more clearly from the outset.
Glad to help! And hey, clarifying our ideas is half of what discussion is for!
I’d love to see a top-level post on ideas for making this happen, since I think you’re right, even though safety in current AI systems is very different from the problems we would face with AGI-level systems.
Does this remind you of what I’m trying to get at? bc it sure does, to me:
https://twitter.com/ESYudkowsky/status/1537842203543801856?s=20&t=5THtjV5sUU1a7Ge1-venUw
but I’m prob going to stay in the “dumb questions” area and not comment :)
ie. “the feeling I have when someone tries to teach me that human-safety is orthogonal to AI-Capability—in a real implementation, they’d be correlated in some way”