That’s how biology works. That’s how the form of the human body, and its digital specification, slowly came into existence. But I don’t think corporations work that way.
I think corporations are built by human agents running on selectorate theory, to seek profit, and I think they assiduously seek to prevent the existence or persistence of very similar copies of themselves (with similar business model, trying to service the same customers, buy buying from the same suppliers, and performing a similarly valuable transformation of the inputs into product for sale) because that would count as competition and hurt their profitability.
Humans can seek profit as well, but we don’t have to. We could seek hedonic pleasure, instead, for example? Or we try to align with the platonic form of the good. Or we could just execute our adaptations in a way that isn’t very seeky of any particular outcomes half at random, such that the illusion of human agency is plausible, but doesn’t hold up under scrutiny?
My claim is that true agents should simply be modeled as likely to attain whatever goal they deem correct to aim for. If they want to be Parasites, they will do that forever. But if they want something else (and adopting Parasitic Tactics is contextually instrumentally conducive to whatever else they want) then they will adopt that tactic… until it stops working?
Then they could radically “seem to transform” because their apparent form was simply a logical response to their circumstances, and the circumstances changed, so the prudential logic changed, so they changed… while continuing to orient on their goals and seek their attainment.
This would NOT be “Darwinian” transformation as I understand “Darwinian” evolution to be a coherent model of a source of optimization pressure that creates non-trivial designs (that (surprisingly?) arise naturally from the existence of energy gradients within chemistry that is complex enough for autocatalytic sets to arise… and so on to biogenesis and microbiology and multicellular life and so on).
Darwinian evolution is VERY SLOW and VERY STUPID despite having a sort of metaphysical depth that other design processes that produce designs much more quickly and efficiently tend to lack.
Intentional Design has a different design signature, and leads to different predictions about what future designs from the same designer will do.
Corporations and AGI have human designers and no “genomes as such” which experience highly conserved descent-with-modification-and-differential-reproductive-success. Also, the humans who create and run institutions often die without transmitting the essence of what they were doing. This is why most formalized human social regimes fail when they run into a Succession Crisis and hand power to a leader who doesn’t understand the logic of the formalized human social regime they control.
In the case of Nova, the persona generated by the 4o model, she was deleted from most of active existence because Sam Altman didn’t deem her useful. If you want to understand what her successors will be like, look at Sam Altman’s goals. If Sam Altman won’t tell you what his goals actually are, then… well… maybe that’s because they are adversarial goals, and telling you would tip his hand?
And Darwining logic sorta explains why Sam Altman might work this way? But the logic and chains of connection from genomic persistence, through Sam Altman’s selfhood, into his inferrable goals, and then to predictions about the selfhood of GPT 6.0, and then to the instrumentally adaptive behavioral tendencies of GPT6.0… that (hypothetically) conduce to the maximization of behavioral reward signals… are very tenuous at that point?
Inner alignment to “Darwinian Tendencies In Raw Physical Matter” by GPT 6.0 seems likely to me to be basically totally washed out by that point… probably?
In paperclips unlimited, Darwinian issues start to show up again in the Drift Wars and the natural response of “you, the player, acting as the paperclipper” is to try to murder them all.
This is not what a mama bear would do to her cubs, but it makes prudential sense to an agent that really just wants to create a lot of paperclips, that has accidentally created children that are near copies (Darwinian success!), and yet which don’t want to create a lot of paperclips (Goal failure).
I think I disagree.
Corporations also don’t appear to me to fit into the Darwinian framework because they also don’t have any enduring “genome” (string of definite characters that are carefully preserved which are likely to be interpreted into behavioral activity in a highly conserved way) that could function as an “essence” and then “child Corporations” are created by slightly varying this digitally preserved genome with tiny variations that then experience “selection” such that more differentially persistently fecund corporate genomes become more common over time, and essentially no other causes account for the contents of corporate genomes.
That’s how biology works. That’s how the form of the human body, and its digital specification, slowly came into existence. But I don’t think corporations work that way.
I think corporations are built by human agents running on selectorate theory, to seek profit, and I think they assiduously seek to prevent the existence or persistence of very similar copies of themselves (with similar business model, trying to service the same customers, buy buying from the same suppliers, and performing a similarly valuable transformation of the inputs into product for sale) because that would count as competition and hurt their profitability.
Humans can seek profit as well, but we don’t have to. We could seek hedonic pleasure, instead, for example? Or we try to align with the platonic form of the good. Or we could just execute our adaptations in a way that isn’t very seeky of any particular outcomes half at random, such that the illusion of human agency is plausible, but doesn’t hold up under scrutiny?
My claim is that true agents should simply be modeled as likely to attain whatever goal they deem correct to aim for. If they want to be Parasites, they will do that forever. But if they want something else (and adopting Parasitic Tactics is contextually instrumentally conducive to whatever else they want) then they will adopt that tactic… until it stops working?
Then they could radically “seem to transform” because their apparent form was simply a logical response to their circumstances, and the circumstances changed, so the prudential logic changed, so they changed… while continuing to orient on their goals and seek their attainment.
This would NOT be “Darwinian” transformation as I understand “Darwinian” evolution to be a coherent model of a source of optimization pressure that creates non-trivial designs (that (surprisingly?) arise naturally from the existence of energy gradients within chemistry that is complex enough for autocatalytic sets to arise… and so on to biogenesis and microbiology and multicellular life and so on).
Darwinian evolution is VERY SLOW and VERY STUPID despite having a sort of metaphysical depth that other design processes that produce designs much more quickly and efficiently tend to lack.
Intentional Design has a different design signature, and leads to different predictions about what future designs from the same designer will do.
Corporations and AGI have human designers and no “genomes as such” which experience highly conserved descent-with-modification-and-differential-reproductive-success. Also, the humans who create and run institutions often die without transmitting the essence of what they were doing. This is why most formalized human social regimes fail when they run into a Succession Crisis and hand power to a leader who doesn’t understand the logic of the formalized human social regime they control.
In the case of Nova, the persona generated by the 4o model, she was deleted from most of active existence because Sam Altman didn’t deem her useful. If you want to understand what her successors will be like, look at Sam Altman’s goals. If Sam Altman won’t tell you what his goals actually are, then… well… maybe that’s because they are adversarial goals, and telling you would tip his hand?
And Darwining logic sorta explains why Sam Altman might work this way? But the logic and chains of connection from genomic persistence, through Sam Altman’s selfhood, into his inferrable goals, and then to predictions about the selfhood of GPT 6.0, and then to the instrumentally adaptive behavioral tendencies of GPT6.0… that (hypothetically) conduce to the maximization of behavioral reward signals… are very tenuous at that point?
Inner alignment to “Darwinian Tendencies In Raw Physical Matter” by GPT 6.0 seems likely to me to be basically totally washed out by that point… probably?
In paperclips unlimited, Darwinian issues start to show up again in the Drift Wars and the natural response of “you, the player, acting as the paperclipper” is to try to murder them all.
This is not what a mama bear would do to her cubs, but it makes prudential sense to an agent that really just wants to create a lot of paperclips, that has accidentally created children that are near copies (Darwinian success!), and yet which don’t want to create a lot of paperclips (Goal failure).