The efficiency of prizes

Several commenters on SoTW: Be Specific suggest that prizes may not be a good idea because they might be counterproductive. As a counterpoint, Tony Barrett of GCRi pointed me to this piece at NYT blogs:

While awards for past achievement can spur innovation — a significant amount of American investigative journalism might not take place without prizes like the Pulitzer — forward-looking prizes for specific challenges are far more efficient. They are extraordinarily cost-effective. Look at Harrison and his clock: the £20,000 he won was enormous for him, but it was a tiny sum compared to what the British government might have spent to solve the longitude problem by giving grants to various astronomers or cartographers. The fact that prizes pay only for success also makes them attractive to those who finance them, whether they are taxpayers, shareholders or charitable donors.

More important, prizes work where other methods do not. A lot of problems aren’t new — someone has already solved them or has solved something similar. By casting a very wide net, prizes find these people.

Winners tend not to be the people you expect. The principal scientific adviser to the Board of Longitude — someone to be reckoned with, as it was Isaac Newton — had said that the only solutions possible for the longitude problem would come from astronomy. No one thought the answer would be found in a new kind of clock. If the British had spent money on grants to the usual suspects, it would have been wasted. The conventional wisdom is that if you want to solve a molecular biology problem you ask only molecular biologists. Big mistake.

At Harvard Business School, Lakhani led a study of hundreds of scientific problems posted on InnoCentive. These were problems that the laboratories of science-driven companies had mostly failed to solve, which is why they turned to InnoCentive. They found that InnoCentive’s network solved nearly 30 percent of them.

What made for success? InnoCentive asks solvers to check boxes indicating the different scientific fields that interest them. The more diverse the interests of the base of solvers, the more likely the problem was to be solved. The study also found that expertise in the field of the problem actually hurt a solver’s chances. “The further the problem from the solver’s expertise, the more likely they are to solve it,” Lakhini and his co-authors concluded. If the problem fell completely outside a solver’s expertise, that raised his or her chance of success by 10 percent. In addition to being a technical outsider, being a social outsider also helped — women did significantly better than men, perhaps because they tended to be more marginalized in the scientific community. Alph Bingham, one of InnoCentive’s founders, told McKinsey that “you wouldn’t hire” a significant percentage of successful solvers based on their credentials.

This seems bizarre, but it is consistent with scientific history, which shows that innovation occurs when knowledge from one scientific discipline is applied to another. “Innovation happens when someone comes in from a different perspective and breaks a problem open,” says Lakhani. “But rarely do we have mechanisms in place so this happens systematically.” Prizes provide that mechanism.