Yeah, this post is kind of a mix between (first) actually trying to explain/discuss the credit assignment problem, and (second) some far more hand-wavy thoughts relating to myopia / partial agency. I have a feeling that the post was curated based on the first part, whereas my primary goal in writing was actually the second part.
In terms of what I talked about (iirc—not re-reading the post for the sake of this comment), it seems like you’re approaching a problem of how to actually assign credit once you have a model. This is of course an essential part of the problem. I would ask questions like:
Does the proposed formula allow us to construct a mechanism which incentivizes a desirable type of behavior?
Does it encourage honesty?
Does it encourage strategically correcting things toward the best collective outcome (even if dishonestly)?
These are difficult problems. However, my post was more focused on the idea that you need a model at all in order to do credit assignment, which I feel is not obvious to many people.
Yeah, this post is kind of a mix between (first) actually trying to explain/discuss the credit assignment problem, and (second) some far more hand-wavy thoughts relating to myopia / partial agency. I have a feeling that the post was curated based on the first part, whereas my primary goal in writing was actually the second part.
In terms of what I talked about (iirc—not re-reading the post for the sake of this comment), it seems like you’re approaching a problem of how to actually assign credit once you have a model. This is of course an essential part of the problem. I would ask questions like:
Does the proposed formula allow us to construct a mechanism which incentivizes a desirable type of behavior?
Does it encourage honesty?
Does it encourage strategically correcting things toward the best collective outcome (even if dishonestly)?
These are difficult problems. However, my post was more focused on the idea that you need a model at all in order to do credit assignment, which I feel is not obvious to many people.