Hi there, it seems to me that this dispute cannot be settled in the abstract.
The ratio of gains made by human-led ai agents vs rogue ai agents will depend on parameters that are difficult to bound sufficiently precisely to determine the trend: the rate of defection (say fixed for simplicity, or assuming agents already understand the dynamics perfectly so they start at a stable value), the gains per unit of compute for rogue/human-led agents (higher for human-led agents as lilkim2025 says, at least in the regime where agents are not superhuman), the fractions of gains devoted to survival and replication in each case (likely lower by a factor of ~1 to ~10 for human-led agents), and the removal hazards (lower for human-led agents as lilkim2025 suggested).
Because the defection rate could be significant (eg ~1/10 to ~1), the standard toy model for this dynamic gives two possible trends: rogue agents eventually dominate the market or the ratio mentioned earlier goes to a fixed finite value.
Vincent_Bagayoko
Karma: 14
I think this is a great idea! One can also imagine someone who has never heard about any of this would first ask their chatbot for clarifications on the corresponding event/theme, and receive a rather convincing, more elaborate answer.
I’m not sure this seed suffices to grow an actual concern for many (if people are already exposed to “AI risk is hype”), but it does carry the strength that it would be coming from the labs themselves, something which causes surprise and eagerness to see what this is about.
Do you have any idea how one could push for this?