You “would have expected” that and you would be wrong.
Doesn’t matter if it’s just 32 bits worth of connections—it’s 32 bits worth of connections that aren’t currently present in the model. Nothing fundamental stops them from being present. It’s just that no one burned them in with training, so they aren’t there.
Finding all the ways to generalize from improved prompts and scaffolding into improved models isn’t at all trivial. And people aren’t at the point of “search and refinement at scale” yet.
Downvoted because your string of comments here feel weirdly aggro, and don’t feel like they are really engaging with what I’m saying. (Like, yes, I am saying I am confused and presumably wrong about something. You seem to want to go hard on saying “yes you’re wrong!” and I’m like “yes, I know?”)
You “would have expected” that and you would be wrong.
Doesn’t matter if it’s just 32 bits worth of connections—it’s 32 bits worth of connections that aren’t currently present in the model. Nothing fundamental stops them from being present. It’s just that no one burned them in with training, so they aren’t there.
Finding all the ways to generalize from improved prompts and scaffolding into improved models isn’t at all trivial. And people aren’t at the point of “search and refinement at scale” yet.
Downvoted because your string of comments here feel weirdly aggro, and don’t feel like they are really engaging with what I’m saying. (Like, yes, I am saying I am confused and presumably wrong about something. You seem to want to go hard on saying “yes you’re wrong!” and I’m like “yes, I know?”)