Great comment Ajsja, you hit the mark. Two small comments:
(i) The ‘correct’, ‘mathematically righteous’ way to calculate credit is through a elaboration of counterfactual impact: the Shapley value. I believe it captures the things you want from credit that you write here.
(ii) On Turing being on the scent of information theory - I find this quote not that compelling. The idea of information as a logarithmic quantity was important but only a fraction of what Shannon did. In general, I agree with Schmidthuber’s assesment that Turing’ scientific stature is a little overrated.
A better comparison would probably be Ralph Hartley pioneered information-theoretic ideas (see e.g. the Shannon-Hartley theorem). I’m sure you know more about the history here than I do.
I’m certain one could write an entire book about the depth, significance and subtlety of Claude Shannon’s work. Perennially underrated.
Shapley seems like quite an arbitrary choice (why uniform over all coalitions?).
I think the actually mathematically right thing is just EDT/UDT, though this doesn’t imply a clear notion of credit. (Maximizing shapley yields crazy results.)
Unfortunately, I don’t think there is a correct notion of credit.
Averaging over all coalitions seems quite natural to me; it averages out the “incidental, contigent, unfair” factor of who got in what coalition first. But tastes may differ.
Shapley value has many other good properties nailing it down as a canonical way to allocate credit.
The Shapley value is uniquely determined by simple properties.
These properties:
Property 1: Sum of the values adds up to the total value (Efficiency)
Property 2: Equal agents have equal value (Symmetry)
Property 3: Order indifference: it doesn’t matter which order you go in (Linearity). Or, in other words, if there are two steps, Value(Step1 + Step2) = Value(Step1) + Value(Step2).
And an extra property:
Property 4: Null-player (if in every world, adding a person to the world has no impact, the person has no impact). You can either take this as an axiom, or derive it from the first three properties.
In the context of scientific contributions, one might argue that property 1 & 2 are very natural, axiomatic while property 3 is merely very reasonable.
I agree Shapley value per se isn’t the answer to all questions of credit. For instance, the Shapley value is not compositional: merging players into a single player doesn’t preserve Shapley values.
Nevertheless, I feel it is a very good idea that has many or all properties people want when they talk about a right notion of credit.
I don’t know what you mean by UDT/EDT in this context—I would be super curious if you could elucidate! :)
What do you mean by maximizing Shapley value gives crazy results? (as I point out above, Shapley value isn’t the be all and end all of all questions of credit and in e.g. hierarchichal composition of agency isn’t well-behaved).
Great comment Ajsja, you hit the mark. Two small comments:
(i) The ‘correct’, ‘mathematically righteous’ way to calculate credit is through a elaboration of counterfactual impact: the Shapley value. I believe it captures the things you want from credit that you write here.
(ii) On Turing being on the scent of information theory - I find this quote not that compelling. The idea of information as a logarithmic quantity was important but only a fraction of what Shannon did. In general, I agree with Schmidthuber’s assesment that Turing’ scientific stature is a little overrated.
A better comparison would probably be Ralph Hartley pioneered information-theoretic ideas (see e.g. the Shannon-Hartley theorem). I’m sure you know more about the history here than I do.
I’m certain one could write an entire book about the depth, significance and subtlety of Claude Shannon’s work. Perennially underrated.
Shapley seems like quite an arbitrary choice (why uniform over all coalitions?).
I think the actually mathematically right thing is just EDT/UDT, though this doesn’t imply a clear notion of credit. (Maximizing shapley yields crazy results.)
Unfortunately, I don’t think there is a correct notion of credit.
Averaging over all coalitions seems quite natural to me; it averages out the “incidental, contigent, unfair” factor of who got in what coalition first. But tastes may differ.
Shapley value has many other good properties nailing it down as a canonical way to allocate credit.
Quoting from nunoSempere’s article:
The Shapley value is uniquely determined by simple properties.
These properties:
Property 1: Sum of the values adds up to the total value (Efficiency)
Property 2: Equal agents have equal value (Symmetry)
Property 3: Order indifference: it doesn’t matter which order you go in (Linearity). Or, in other words, if there are two steps, Value(Step1 + Step2) = Value(Step1) + Value(Step2).
And an extra property:
Property 4: Null-player (if in every world, adding a person to the world has no impact, the person has no impact). You can either take this as an axiom, or derive it from the first three properties.
In the context of scientific contributions, one might argue that property 1 & 2 are very natural, axiomatic while property 3 is merely very reasonable.
I agree Shapley value per se isn’t the answer to all questions of credit. For instance, the Shapley value is not compositional: merging players into a single player doesn’t preserve Shapley values.
Nevertheless, I feel it is a very good idea that has many or all properties people want when they talk about a right notion of credit.
I don’t know what you mean by UDT/EDT in this context—I would be super curious if you could elucidate! :)
What do you mean by maximizing Shapley value gives crazy results? (as I point out above, Shapley value isn’t the be all and end all of all questions of credit and in e.g. hierarchichal composition of agency isn’t well-behaved).
Say more / references?