Improved personality is indeed a real, important improvement in the models, but (compared to traditional pre-training scaling) it feels like more of a one-off “unhobbling” than something we should expect to continue driving improved performance in the future. Going from pure next-token-predictors to chatbots with RLHF was a huge boost in usefulness. Then, going from OpenAI’s chatbot personality to Claude’s chatbot personality was a noticeable (but much smaller) boost. But where do we go from here? I can’t really imagine a way for Anthropic to improve Claude’s personality by 10x or 100x (whatever that would even mean). Versus I can imagine scaling RL to improve a reasoning model’s math skills by 100x.
Interesting point about personality improvements being a “one-off unhobbling” with diminishing returns. But I wonder if this reflects a measurement bias rather than an actual capability ceiling: we have clear benchmarks for evaluating math skills—it’s easy to measure 100x improvement when a model goes from solving basic algebra to proving novel theorems. But how do we quantify personality improvements? There’s a vast gap between “helpful but generic” and “perfectly attuned to individual users’ needs, communication styles, and thinking patterns.”
I can imagine future models that feel like they truly understand me personally, anticipate my unstated needs, communicate in exactly my preferred style, and adapt their approach based on my emotional state—something far beyond current implementations. The lack of obvious metrics for these qualities doesn’t mean the improvement ceiling is low, just that we’re not good at measuring them yet. Thoughts?
That’s a good point—the kind of idealized personal life coach / advisor Dario describes in his post “Machines of Loving Grace” is definitely in a sense a personality upgrade over Claude 3.7. But I feel like when you think about it more closely, most of the improvements from Claude to ideal-AI-life-coach are coming from non-personality improvements, like:
having a TON of context about my personal life, interests, all my ongoing projects and relationships, etc
having more intelligence (including reasoning ability, but also fuzzier skills like social / psychological modeling) to bring to bear on brainstorming solutions to problems, or identifying the root cause of various issues, etc. (does the idea of “superpersuasion” load more heavily on superintelligence, or on “superpersonality”? seems like a bit of both; IMO you would at least need considerable intelligence even if it’s somehow mostly tied to personality)
even the gains that I’d definitely count as personality improvements, might not all come primarily from more-tasteful RLHF creating a single, ideal super-personality (like what Claude currently aims for). Instead, an ideal AI advisor product would probably be able to identify the best way of working with a given patient/customer, and tailor its personality to work well with that particular individual. RLHF as practiced today can do this to a limited extent (ie, claude can do things like sense whether a formal vs informal style of reply would be more appropriate, given the context), but I feel like new methods beyond centralized RLHF might be needed to fully customize an AI’s personality to each individual.
Improved personality is indeed a real, important improvement in the models, but (compared to traditional pre-training scaling) it feels like more of a one-off “unhobbling” than something we should expect to continue driving improved performance in the future. Going from pure next-token-predictors to chatbots with RLHF was a huge boost in usefulness. Then, going from OpenAI’s chatbot personality to Claude’s chatbot personality was a noticeable (but much smaller) boost. But where do we go from here? I can’t really imagine a way for Anthropic to improve Claude’s personality by 10x or 100x (whatever that would even mean). Versus I can imagine scaling RL to improve a reasoning model’s math skills by 100x.
Interesting point about personality improvements being a “one-off unhobbling” with diminishing returns. But I wonder if this reflects a measurement bias rather than an actual capability ceiling: we have clear benchmarks for evaluating math skills—it’s easy to measure 100x improvement when a model goes from solving basic algebra to proving novel theorems. But how do we quantify personality improvements? There’s a vast gap between “helpful but generic” and “perfectly attuned to individual users’ needs, communication styles, and thinking patterns.”
I can imagine future models that feel like they truly understand me personally, anticipate my unstated needs, communicate in exactly my preferred style, and adapt their approach based on my emotional state—something far beyond current implementations. The lack of obvious metrics for these qualities doesn’t mean the improvement ceiling is low, just that we’re not good at measuring them yet. Thoughts?
That’s a good point—the kind of idealized personal life coach / advisor Dario describes in his post “Machines of Loving Grace” is definitely in a sense a personality upgrade over Claude 3.7. But I feel like when you think about it more closely, most of the improvements from Claude to ideal-AI-life-coach are coming from non-personality improvements, like:
having a TON of context about my personal life, interests, all my ongoing projects and relationships, etc
having more intelligence (including reasoning ability, but also fuzzier skills like social / psychological modeling) to bring to bear on brainstorming solutions to problems, or identifying the root cause of various issues, etc. (does the idea of “superpersuasion” load more heavily on superintelligence, or on “superpersonality”? seems like a bit of both; IMO you would at least need considerable intelligence even if it’s somehow mostly tied to personality)
even the gains that I’d definitely count as personality improvements, might not all come primarily from more-tasteful RLHF creating a single, ideal super-personality (like what Claude currently aims for). Instead, an ideal AI advisor product would probably be able to identify the best way of working with a given patient/customer, and tailor its personality to work well with that particular individual. RLHF as practiced today can do this to a limited extent (ie, claude can do things like sense whether a formal vs informal style of reply would be more appropriate, given the context), but I feel like new methods beyond centralized RLHF might be needed to fully customize an AI’s personality to each individual.