The post is about what “adulthood” means for goal engines, and where the vector from baby to adulthood points. Current AI safety work is only relevant to a “system that is still sufficiently baby-like”. But we should expect goal engines to be extremely mature. When you are negotiating with a human adult who is trying to maximize their company’s profit, there is no need to study the phenotype of the 3-month embryo that once scaffolded that human.
The post is about what “adulthood” means for goal engines, and where the vector from baby to adulthood points. Current AI safety work is only relevant to a “system that is still sufficiently baby-like”. But we should expect goal engines to be extremely mature. When you are negotiating with a human adult who is trying to maximize their company’s profit, there is no need to study the phenotype of the 3-month embryo that once scaffolded that human.