Hmm, I think we’re seeing right now the pitfalls of terms like “AGI” and “superhuman” in past forecasting. Like, Tyler Cowen keeps saying o3 is AGI, which seems obviously wrong to me, but there’s enough wiggle room in the term that all I can really do is shrug. More specific claims are easier to evaluate in the future. I don’t have a bone to pick with long reports that carefully define lots of terms and make concrete predictions, but on the margin I think there’s too much ephemeral, buzzy conversation about “this model is PhD level” or “this model is human level at task xyz” or what have you. I’m far less interested in when we’ll have a “human level novelist” system, and more in, say, in what year an AI generated book will first be a New York Times bestseller (and that social forces might prevent this is both feature and bug).
Hmm, I think we’re seeing right now the pitfalls of terms like “AGI” and “superhuman” in past forecasting. Like, Tyler Cowen keeps saying o3 is AGI, which seems obviously wrong to me, but there’s enough wiggle room in the term that all I can really do is shrug. More specific claims are easier to evaluate in the future. I don’t have a bone to pick with long reports that carefully define lots of terms and make concrete predictions, but on the margin I think there’s too much ephemeral, buzzy conversation about “this model is PhD level” or “this model is human level at task xyz” or what have you. I’m far less interested in when we’ll have a “human level novelist” system, and more in, say, in what year an AI generated book will first be a New York Times bestseller (and that social forces might prevent this is both feature and bug).