It might be better to make the criteria about cognitive performance vs. computation, but it is harder to define and therefore enforce
Agreed that there’s a lot more detail that would have to be nailed down to do it this way. I think one big advantage to defining it by cognitive performance is to make it clearer to the general public. “Was trained using more than 10^26 FLOPS” doesn’t mean anything at all to most people (and doesn’t relate to capabilities for anyone who hasn’t investigated that exact relationship). “Is smarter than human” is very intuitively clear to most people (I think?) and so it may be easier to coordinate around.
Excellent point. It’s a far better movement slogan. So even if you wanted to turn it into a compute limit, that should be how the goal is framed.
I also wonder about replacing “intelligence” with “competence”. Lots of people now say “intelligent at what? They’ve beaten us at chess forever and that’s fine”. You can do the same thing with competence, but the instinct hasn’t developed. And the simple answer is “competent at taking over the world”.
Agreed that there’s a lot more detail that would have to be nailed down to do it this way. I think one big advantage to defining it by cognitive performance is to make it clearer to the general public. “Was trained using more than 10^26 FLOPS” doesn’t mean anything at all to most people (and doesn’t relate to capabilities for anyone who hasn’t investigated that exact relationship). “Is smarter than human” is very intuitively clear to most people (I think?) and so it may be easier to coordinate around.
Excellent point. It’s a far better movement slogan. So even if you wanted to turn it into a compute limit, that should be how the goal is framed.
I also wonder about replacing “intelligence” with “competence”. Lots of people now say “intelligent at what? They’ve beaten us at chess forever and that’s fine”. You can do the same thing with competence, but the instinct hasn’t developed. And the simple answer is “competent at taking over the world”.
My initial intuition is that “more competent than humans” won’t resonate as much as “smarter than humans” but that’s just a guess.