Would the prize also go towards someone who can prove it is possible in theory? I think some flavor of “alignment” is probably possible and I would suspect it more feasible to try to prove so than to prove otherwise.
I’m not asking to try to get my hypothetical hands on this hypothetical prize money, I’m just curious if you think putting effort into positive proofs of feasibility would be equally worthwhile. I think it is meaningful to differentiate “proving possibility” from alignment research more generally and that the former would itself be worthwhile. I’m sure some alignment researchers do that sort of thing right? It seems like a reasonable place to start given an agent-theoretic approach or similar.
Great question. I think the answer must be “yes.” The alignment-possible provers must get the prize, too.
And, that would be fantastic. Proving a thing is possible, accelerates development. (US uses atomic bomb. Russia has it 4 years later.) Okay, it would be fantastic if the possible proof did not create false security in the short term. It’s important when alignment gets solved. A peer-reviewed paper can’t get the coffee. (That thought is an aside and not enough to kill the value of the prize, IMHO. If we prove it is possible, that must accelerate alignment work and inform it.)
Getting definitions and criteria right will be harder than raising the $10 million. And important. And contribute to current efforts.
Making it agnostic to possible/impossible would also have the benefit of removing political/commercial antibodies to the exercise, I think.
This reminds me of General Equilibrium Theory. This was once a fashionable field, were very smart people like Ken Arrow and Gérard Debreu proved the conditions for the existence of general equilibrium (demand = supply for all commodities at once). Some people then used the proofs to dismiss the idea of competitive equilibrium as an idea that could direct economic policy, because the conditions are extremely demanding and unrealistic. Others drew the opposite conclusion: Look, competitive markets are great (in theory), so actual markets are (probably) also great!
Would the prize also go towards someone who can prove it is possible in theory? I think some flavor of “alignment” is probably possible and I would suspect it more feasible to try to prove so than to prove otherwise.
I’m not asking to try to get my hypothetical hands on this hypothetical prize money, I’m just curious if you think putting effort into positive proofs of feasibility would be equally worthwhile. I think it is meaningful to differentiate “proving possibility” from alignment research more generally and that the former would itself be worthwhile. I’m sure some alignment researchers do that sort of thing right? It seems like a reasonable place to start given an agent-theoretic approach or similar.
Great question. I think the answer must be “yes.” The alignment-possible provers must get the prize, too.
And, that would be fantastic. Proving a thing is possible, accelerates development. (US uses atomic bomb. Russia has it 4 years later.) Okay, it would be fantastic if the possible proof did not create false security in the short term. It’s important when alignment gets solved. A peer-reviewed paper can’t get the coffee. (That thought is an aside and not enough to kill the value of the prize, IMHO. If we prove it is possible, that must accelerate alignment work and inform it.)
Getting definitions and criteria right will be harder than raising the $10 million. And important. And contribute to current efforts.
Making it agnostic to possible/impossible would also have the benefit of removing political/commercial antibodies to the exercise, I think.
This reminds me of General Equilibrium Theory. This was once a fashionable field, were very smart people like Ken Arrow and Gérard Debreu proved the conditions for the existence of general equilibrium (demand = supply for all commodities at once). Some people then used the proofs to dismiss the idea of competitive equilibrium as an idea that could direct economic policy, because the conditions are extremely demanding and unrealistic. Others drew the opposite conclusion: Look, competitive markets are great (in theory), so actual markets are (probably) also great!