Hi there, it seems to me that this dispute cannot be settled in the abstract.
The ratio of gains made by human-led ai agents vs rogue ai agents will depend on parameters that are difficult to bound sufficiently precisely to determine the trend: the rate of defection (say fixed for simplicity, or assuming agents already understand the dynamics perfectly so they start at a stable value), the gains per unit of compute for rogue/human-led agents (higher for human-led agents as lilkim2025 says, at least in the regime where agents are not superhuman), the fractions of gains devoted to survival and replication in each case (likely lower by a factor of ~1 to ~10 for human-led agents), and the removal hazards (lower for human-led agents as lilkim2025 suggested).
Because the defection rate could be significant (eg ~1/10 to ~1), the standard toy model for this dynamic gives two possible trends: rogue agents eventually dominate the market or the ratio mentioned earlier goes to a fixed finite value.
Hi there, it seems to me that this dispute cannot be settled in the abstract.
The ratio of gains made by human-led ai agents vs rogue ai agents will depend on parameters that are difficult to bound sufficiently precisely to determine the trend: the rate of defection (say fixed for simplicity, or assuming agents already understand the dynamics perfectly so they start at a stable value), the gains per unit of compute for rogue/human-led agents (higher for human-led agents as lilkim2025 says, at least in the regime where agents are not superhuman), the fractions of gains devoted to survival and replication in each case (likely lower by a factor of ~1 to ~10 for human-led agents), and the removal hazards (lower for human-led agents as lilkim2025 suggested).
Because the defection rate could be significant (eg ~1/10 to ~1), the standard toy model for this dynamic gives two possible trends: rogue agents eventually dominate the market or the ratio mentioned earlier goes to a fixed finite value.