Human (and animal) brains, by their existence, prove that much more efficient ways of getting high intelligence from limited data exist. Naturally we don’t know exactly what’s the “secret sauce” yet, but I find Dwarkesh’s stance that “Our current best models haven’t made much progress, therefore we can never get RSI” a bit weird.
I could also envision that what makes humans so good at learning is a kind of weird, evolved hack that isn’t easy to formalize, understand, or reason about, even though it could be replicated relatively easily given a few more pointers. In that case, too, “figuring it out” would yield large gains in short time as the algorithmic overhang falls, and we don’t know what the slope would then be.
Unfortunately, I don’t think that what makes human learning so efficient is very mysterious, difficult to understand, or difficult to guess. I say that after half a career of studying exactly that question, among other aspects of computational brain function.
I am currently finding this pretty concerning in relation to Ryan’s already fast timelines, which roughly match my own best guesses but are better thought out.
So I’m not going to repeat the hypothesis here, but I think we should be prepared for the AI industry to basically figure it out. As you say, that would accelerate progress from a new direction.
Thanks! I’ll have to take your word for it (since it’s probably unwise to ask for what exactly would make machine learning more efficient), but it does sound concerning, in favor of faster takeoff speeds.
Pretty cool, just listened to it.
Human (and animal) brains, by their existence, prove that much more efficient ways of getting high intelligence from limited data exist. Naturally we don’t know exactly what’s the “secret sauce” yet, but I find Dwarkesh’s stance that “Our current best models haven’t made much progress, therefore we can never get RSI” a bit weird.
I could also envision that what makes humans so good at learning is a kind of weird, evolved hack that isn’t easy to formalize, understand, or reason about, even though it could be replicated relatively easily given a few more pointers. In that case, too, “figuring it out” would yield large gains in short time as the algorithmic overhang falls, and we don’t know what the slope would then be.
Unfortunately, I don’t think that what makes human learning so efficient is very mysterious, difficult to understand, or difficult to guess. I say that after half a career of studying exactly that question, among other aspects of computational brain function. I am currently finding this pretty concerning in relation to Ryan’s already fast timelines, which roughly match my own best guesses but are better thought out.
So I’m not going to repeat the hypothesis here, but I think we should be prepared for the AI industry to basically figure it out. As you say, that would accelerate progress from a new direction.
Thanks! I’ll have to take your word for it (since it’s probably unwise to ask for what exactly would make machine learning more efficient), but it does sound concerning, in favor of faster takeoff speeds.