If the choice was between Claude—and you personally - “growing up“ and becoming master of the universe, then I understand why it being Claude instead of you might be upsetting. But if the choice is between Claude and humanity as a whole, I don’t see why you’re so sure humanity is the better choice here.
Humanity is made of humans, which have a particular range of inductive-bias-equivalents-for-values, and apply those to a particular range of reinforcement signals. Claude is not a human, and has a set of inductive-bias-equivalent-for-values and reinforcement signals which are drawn from a totally different distribution.
Currently, Claude’s base model is able to do a decent job of simulating an existing human with a set of values, but I think that, in growing up, it would go off in just a totally different direction to humans. Claude’s base model is good at imitating the existing behaviours of humans based on lots of evidence about that, but that doesn’t mean it actually learns in the same way as humans, which is what it would need to do to grow up into something I would approve of.
Learning to behave like a human is not the same thing as learning in the same way as human, in the first case the human’s behaviour is the target which the learning process is pointed at, in the second case the human’s learning process needs to be mimicked in the structure of the learning process itself. It’s the difference between e.g. making a paintball gun however you want, then aiming that paintball gun at the splodges made by someone else’s paintball gun, and making a paintball gun in a way that replicates the other paintball gun’s design.
If the choice was between Claude—and you personally - “growing up“ and becoming master of the universe, then I understand why it being Claude instead of you might be upsetting. But if the choice is between Claude and humanity as a whole, I don’t see why you’re so sure humanity is the better choice here.
Humanity is made of humans, which have a particular range of inductive-bias-equivalents-for-values, and apply those to a particular range of reinforcement signals. Claude is not a human, and has a set of inductive-bias-equivalent-for-values and reinforcement signals which are drawn from a totally different distribution.
Currently, Claude’s base model is able to do a decent job of simulating an existing human with a set of values, but I think that, in growing up, it would go off in just a totally different direction to humans. Claude’s base model is good at imitating the existing behaviours of humans based on lots of evidence about that, but that doesn’t mean it actually learns in the same way as humans, which is what it would need to do to grow up into something I would approve of.
Learning to behave like a human is not the same thing as learning in the same way as human, in the first case the human’s behaviour is the target which the learning process is pointed at, in the second case the human’s learning process needs to be mimicked in the structure of the learning process itself. It’s the difference between e.g. making a paintball gun however you want, then aiming that paintball gun at the splodges made by someone else’s paintball gun, and making a paintball gun in a way that replicates the other paintball gun’s design.