Wolfgang Spohn develops the concept of a “dependency equilibrium” based on a similar notion of evidential best response (Spohn 2007, 2010). A joint probability distribution P is a dependency equilibrium if all actions of all players that have positive probability are evidential best responses. In case there are actions with zero probability, one evaluates a sequence (P(i))i∈N of joint probability distributions such that limi→∞P(i)=P and P(i)(a)≠0 for all actions a and i∈N. Using your notation of a probability matrix and a utility matrix, the expected utility of an action aj is then defined as the limit of the conditional expected utilities, limi→∞UjP(i)j|P(i)j| (which is defined for all actions). Say P is a probability matrix with only one zero column, Pj. It seems that you can choose an arbitrary nonzero vector Qj, |Qj|=1 to construct, e.g., a sequence of probability matrices (i−1iP+[0,…,0,1iQj,0,…,0])i∈N. The expected utilities in the limit for all other actions and the actions of the opponent shouldn’t be influenced by this change. So you could choose Qj as the standard vector ei where i is an index such that Uj,i=minUj. The expected utility of aj would then be minUj. Hence, this definition of best response in case there are actions with zero probability probably coincides with yours (at least for actions with positive probability—Spohn is not concerned with the question of whether a zero probability action is a best response or not).
The whole thing becomes more complicated with several zero rows and columns, but I would think it should be possible to construct sequences of distributions which work in that case as well.
Wolfgang Spohn develops the concept of a “dependency equilibrium” based on a similar notion of evidential best response (Spohn 2007, 2010). A joint probability distribution P is a dependency equilibrium if all actions of all players that have positive probability are evidential best responses. In case there are actions with zero probability, one evaluates a sequence (P(i))i∈N of joint probability distributions such that limi→∞P(i)=P and P(i)(a)≠0 for all actions a and i∈N. Using your notation of a probability matrix and a utility matrix, the expected utility of an action aj is then defined as the limit of the conditional expected utilities, limi→∞UjP(i)j|P(i)j| (which is defined for all actions). Say P is a probability matrix with only one zero column, Pj. It seems that you can choose an arbitrary nonzero vector Qj, |Qj|=1 to construct, e.g., a sequence of probability matrices (i−1iP+[0,…,0,1iQj,0,…,0])i∈N. The expected utilities in the limit for all other actions and the actions of the opponent shouldn’t be influenced by this change. So you could choose Qj as the standard vector ei where i is an index such that Uj,i=minUj. The expected utility of aj would then be minUj. Hence, this definition of best response in case there are actions with zero probability probably coincides with yours (at least for actions with positive probability—Spohn is not concerned with the question of whether a zero probability action is a best response or not).
The whole thing becomes more complicated with several zero rows and columns, but I would think it should be possible to construct sequences of distributions which work in that case as well.