# rk comments on AI development incentive gradients are not uniformly terrible

• Comment here if there are maths problems

• I think your solution to “reckless rivals” might be wrong? I think you mistakenly put a multiplier of q instead of a p on the left-hand side of the inequality. (The derivation of the general inequality checks out, though, and I like your point about discontinuous effects of capacity investment when you assume that the opponent plays a known pure strategy.)

I’ll use slightly different notation from yours, to avoid overloading p and q. (This ends up not mattering because of linearity, but eh.) Let be the initial probabilities for winning and safety|winning. Let be the capacity variable, and without loss of generality let start at and end at . Then and . So , so . And , so .

Therefore, the left-hand side of the inequality, , equals . At the initial point , this simplifies to .

Let’s assume . The relative safety of the other project is , which at simplifies to .

Thus we should commit more to capacity when , or , or . This is a little weird, but makes a bit more intuitive sense to me than or mattering.

• Yes, you’re quite right!

The intuition becomes a little clearer when I take the following alternative derivation:

Let us look at the change in expected value when I increase my capabilities. From the expected value stemming from worlds where I win, we have . For the other actor, their probability of winning decreases at a rate that matches my increase in probability of winning. Also, their probability of deploying a safe AI doesn’t change. So the change in expected value stemming fro m worlds where they win is .

We should be indifferent to increasing capabilities when these sum to 0, so .

Let’s choose our units so . Then, using the expressions for from your comment, we have .

Dividing through by we get . Collecting like terms we have and thus . Substituting for we have and thus

• Oh wait, yeah, this is just an example of the general principle “when you’re optimizing for xy, and you have a limited budget with linear costs on x and y, the optimal allocation is to spend equal amounts on both.”

Formally, you can show this via Lagrange-multiplier optimization, using the Lagrangian . Setting the partials equal to zero gets you , and you recover the linear constraint function . So . (Alternatively, just optimizing works, but I like Lagrange multipliers.)

In this case, we want to maximize , which is equivalent to optimizing . Let’s define , so we’re optimizing .

Our constraint function is defined by the tradeoff between and . , so . , so .

Rearranging gives the constraint function . This is indeed linear, with a total ‘budget’ of .5 and a p-coefficient of 1. So by the above theorem we should have .