I’m more trying to operationalize “interesting novel concept.” (But, it does look like we had approximately this conversation before and I’ll try to reread first. I think basically you said “they generate a novel concept that hadn’t been generated before and also people go on to use that concept in industry/science”, does that sound right?)
Part of what brought me here was remembering you saying:
My guess is that all or very nearly all human children have all or nearly all the intelligence juice. We just, like, don’t appreciate how much a child is doing in constructing zer world.
And wanting an example of thing that’s more like “what’s something that’d make you go ‘okay, this was in fact as smart as a four year old’” (and therefore either the end is nigh, or, we’re about to learn that children in fact did not have nearly all the intelligence juice.”)
I’ll try to think about some bets for ~1 year from now.
I think basically you said “they generate a novel concept that hadn’t been generated before and also people go on to use that concept in industry/science”, does that sound right?
Yeah, basically. I’m trying to be concrete here, and just saying “their intellectual output could be judged like human intellectual output is judged”.
And wanting an example of thing that’s more like “what’s something that’d make you go ‘okay, this was in fact as smart as a four year old’” (and therefore either the end is nigh, or, we’re about to learn that children in fact did not have nearly all the intelligence juice.”)
It’s a good question but it’s hard because that stuff looks from the outside like mostly pretty easy tasks. The way in which it is not easy is the way in which it is not “a task”. I guess, “very sample efficient learning” would be a concrete thing that 4yos do.
Nativization of a pidgin into a creole language might be an example, especially given that it seems to be largely underwritten by the cognitive plasticity of the linguistic developmental window.
A creole is believed to arise when a pidgin, developed by adults for use as a second language, becomes the native and primary language of their children – a process known as nativization.
Given that Opus 4.6 fails on very basic Classical Greek exercises (evidence towards “jaggedness”/bad “OOD generalization” even on very simple (though knowledge-heavy) tasks), I would be very surprised if it managed to successfully do something as unusual/OOD as creolizing a pidgin. It might also be very difficult to train it to do so, as it’s a very open-ended thing, and thus it’s very unclear how to specify a reward, and I would guess there isn’t much data on the internet that could be used for training.
Can you give an example of a thing that you’d be surprised if an AI did in the next, say, 1.5 years?
Kill everyone? I’d be pretty surprised, like 1 in 100 or 200 surprised or something like that.
Generating interesting novel concepts on par with humans? See https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce?commentId=dqbLkADbJQJi6bFtN
See also https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce?commentId=HSqkp2JZEmesubDHD#HSqkp2JZEmesubDHD
Now, would you list something impressive that you do expect an AI to do in the next 1.5 years (that I might not say)?
I’m more trying to operationalize “interesting novel concept.” (But, it does look like we had approximately this conversation before and I’ll try to reread first. I think basically you said “they generate a novel concept that hadn’t been generated before and also people go on to use that concept in industry/science”, does that sound right?)
Part of what brought me here was remembering you saying:
And wanting an example of thing that’s more like “what’s something that’d make you go ‘okay, this was in fact as smart as a four year old’” (and therefore either the end is nigh, or, we’re about to learn that children in fact did not have nearly all the intelligence juice.”)
I’ll try to think about some bets for ~1 year from now.
Yeah, basically. I’m trying to be concrete here, and just saying “their intellectual output could be judged like human intellectual output is judged”.
It’s a good question but it’s hard because that stuff looks from the outside like mostly pretty easy tasks. The way in which it is not easy is the way in which it is not “a task”. I guess, “very sample efficient learning” would be a concrete thing that 4yos do.
Nativization of a pidgin into a creole language might be an example, especially given that it seems to be largely underwritten by the cognitive plasticity of the linguistic developmental window.
Given that Opus 4.6 fails on very basic Classical Greek exercises (evidence towards “jaggedness”/bad “OOD generalization” even on very simple (though knowledge-heavy) tasks), I would be very surprised if it managed to successfully do something as unusual/OOD as creolizing a pidgin. It might also be very difficult to train it to do so, as it’s a very open-ended thing, and thus it’s very unclear how to specify a reward, and I would guess there isn’t much data on the internet that could be used for training.