I also buy the econ story here (and, per Ruby, I’m somewhat pleasantly surprised by the amount of reviewing activity given this).
General observation suggests that people won’t find writing reviews that intrinsically motivating (compare to just writing posts, which all the authors are doing ‘for free’ with scant chance of reward, also compare to academia—I don’t think many academics find peer review/refereeing one of the highlights of their job). With apologies for the classic classical econ joke, if reviewing was so valuable, how come people weren’t doing it already? [It also looks like ~25%? of reviews, especially the most extensive, are done by the author on their own work].
If we assume there’s little intrinsic motivation (I’m comfortably in the ‘you’d have to pay me’ camp), the money doesn’t offer that much incentive. Given Rudy’s numbers suppose each of the 82 reviews takes an average of 45 minutes or so (factoring in (re)reading time and similar). If the nomination money is ~roughly allocated by person time spent, the marginal expected return of me taking an hour to review is something like $40. Facially, this isn’t too bad an hourly rate, but the real value is significantly lower:
The ‘person-time lottery’ model should not be denominated by observed person-time so far, but one’s expectation how much will be spent in total once reviewing finishes, which will be higher (especially conditioned on posts like this).
It’s very unlikely the reward is going to allocated proportionately to time spent (/some crude proxy thereof like word count). Thus the EV would be discounted by whatever degree of risk aversion one has (I expect the modal ‘payout’ for a review to be $0).
Opaque allocation also incurs further EV-reducing uncertainty, but best guesses suggest there will be Pareto-principle/tournament dynamic game dynamics, so those with (e.g.) reasons to believe they’re less likely to impress the mod team’s evaluation of their ‘pruning’ have strong reasons to select themselves out.
The malaria story has fair face validity if one observes the wider time series (e.g.). Further, the typical EA ‘picks’ for net distribution are generally seen as filling around the edges of the mega-distributors.
FWIW: I think this discussion would be clearer if framed in last-dollar terms.
If Gates et al. are doing something like last dollar optimisation, trying to save as many lives as they can allocating across opportunities both now and in the future, leaving the right now best marginal interventions on the table would imply they expect to exhaust their last dollar on more cost-effective interventions in the future.
This implies the right now marginal price should be higher than the (expected) last dollar cost effectiveness (if not, it should be reallocating some of the ‘last dollars’ to interventions right now). Yet this in turn does not imply we should see 50Bn of marginal price lifesaving lying around right now. So it seems we can explain Gates et al. not availing themselves of the (non-existent) opportunity to (say) halve communicable diseases for 2Bn a year worldwide (extrapolating from the right now marginal prices) without the right now marginal price being lied about or manipulated. (Obviously, even if we forecast the Gates et al. last dollar EV to be higher than the current marginal price, we might venture alternative explanations of this discrepancy besides them screwing us.)