Infra-Bayesian physicalism: proofs part II

[Vanessa Kosoy]

This post is an appendix to “Infra-Bayesian physicalism: a formal theory of naturalized induction”.

Proposition 2.16: Consider some ${\Gamma }_0$, ${\Gamma }_1$, $\Phi$ and $\Theta \in {\square }^c({\Gamma }_0\times {\Gamma }_1\times \Phi )$. Define $\pi :{\text{el}}^{{\Gamma }_0\times {\Gamma }_1}\to {\text{el}}^{{\Gamma }_0}$ by $\pi (y,z,\alpha ):=(y,\{y^{\prime }\in {\Gamma }_0\mid (y^{\prime },z)\in \alpha \left\}\right)$. Then, $$(\pi \times {\text{id}}_{\Phi }{)}_{\ast }{\text{Br}}_{{\Gamma }_0\times {\Gamma }_1}(\Theta )\subseteq {\text{pr}}_{{\text{el}}^{{\Gamma }_0}\times \Phi }{\text{Br}}_{{\Gamma }_0}(\Theta )\subseteq {\text{Br}}_{{\Gamma }_0}({\text{pr}}_{{\Gamma }_0\times \Phi }\Theta )$$

We can prove the second subset inclusion directly as a corollary of Proposition 2.10, just let the $t$ of Proposition 2.10 be the projection function ${\Gamma }_1\times \Phi \to \Phi$, so that just leaves the first subset inclusion direction. If you’ve seen the proofs so far, you know we do a thing where we try to show subset inclusion with expectations of functions and inequalities instead. And that the proofs all proceed by transforming the expectations until we get a maximum over contribution expectation values, and that’s always where the hard part of proving the inequalities shows up. So, let’s just get that part over with, an interested reader can work it out with previous proofs as a guide. Unusually, we’ll be keeping track of identity functions here.

Plugging in some $f$, and doing our usual activities to get every term into the appropriate form, we can get this result if we manage to show that $$\underset{{\theta }^{\prime }\in {\text{Br}}_{{\Gamma }_0\times {\Gamma }_1}\left(\Theta \right)}{max}(\pi \times {\text{id}}_{\Phi }{)}_{\ast }{\theta }^{\prime }(\lambda y_0{\alpha }_{0}x.f(y_0,x,{\alpha }_0))$$ $$\le \underset{\theta \in {\text{Br}}_{{\Gamma }_0}\left(\Theta \right)}{max}{\text{pr}}_{{\text{el}}^{{\Gamma }_0}\times \Phi }\theta (\lambda y_0{\alpha }_{0}x.f(y_0,x,{\alpha }_0\left)\right)$$ So, to establish this, we’ll show that, given some ${\theta }^{\prime }\in {\text{Br}}_{{\Gamma }_0\times {\Gamma }_1}\left(\Theta \right)$, we have $(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }({\theta }^{\prime })\in {\text{Br}}_{{\Gamma }_0}(\Theta )$, and that $${\text{pr}}_{{\text{el}}^{{\Gamma }_0}\times \Phi }\left(\right(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }{\theta }^{\prime })=(\pi \times {\text{id}}_{\Phi }{)}_{\ast }{\theta }^{\prime }$$ because, if we show that, then it means that ${\text{Br}}_{{\Gamma }_0}\left(\Theta \right)$ is a rich enough set for the right-hand side of the equation to match anything the left-hand-side can put out.

First off, $${\text{pr}}_{{\text{el}}^{{\Gamma }_0}\times \Phi }\left(\right(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }{\theta }^{\prime })=(\pi \times {\text{id}}_{\Phi }{)}_{\ast }{\theta }^{\prime }$$ is pretty trivial to show. The only difference between the two processes is that the ${\Gamma }_1$ coordinate of ${\theta }^{\prime }$ is discarded immediately on the right-hand-side, and it’s preserved for one step and then discarded on the second step for the left-hand-side.

Now for our inequality of interest. Let ${\theta }^{\prime }\in {\text{Br}}_{{\Gamma }_0\times {\Gamma }_1}\left(\Theta \right)$, and we’re trying to show that $$(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }({\theta }^{\prime })\in {\text{Br}}_{{\Gamma }_0}(\Theta )$$ First off, showing the support condition for $(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }({\theta }^{\prime })$ which is somewhat nontrivial this time around. We start off with a guarantee that $(y_0,y_1)\in \alpha$. This happens iff $$y_0\in \left\{y_0^{\prime }|\right(y_0^{\prime },y_1)\in \alpha \}=\pi (y_0,y_1,\alpha {)}_{2^{{\Gamma }_0}}$$ And so, we get that $y_0\in {\alpha }_0$ is guaranteed for that pushforward, support condition established.

endofunction condition time. It’s important to remember that we want to treat ${\text{el}}^{{\Gamma }_0}$ as the computation side of things, and ${\Gamma }_1\times \Phi$ as the environment side of things, for our bridge transform we’re working with.$s:{\Gamma }_0\to {\Gamma }_0$ and $g:{\Gamma }_0\times {\Gamma }_1\times \Phi \to [0,1]$. Begin. $$(\pi \times {\text{id}}_{{\Gamma }_1\times \Phi }{)}_{\ast }{\theta }^{\prime }(\lambda y_0{y}_1{\alpha }_{0}x.{\chi }_{s\left(y_0\right)\in {\alpha }_0}g\left(s\right(y_0),y_1,x))$$ $$={\theta }^{\prime }(\lambda y_0{y}_1\alpha x.{\chi }_{s\left(y_0\right)\in \pi (y_0,\alpha ,y_1{)}_{2^{{\Gamma }_0}}}g(s\left(y_0\right),y_1,x\left)\right)$$ Let’s unpack precisely what that set is. $$={\theta }^{\prime }(\lambda y_0{y}_1\alpha x.{\chi }_{s\left(y_0\right)\in \left\{y_0^{\prime }|\right(y_0^{\prime },y_1)\in \alpha \}}g(s\left(y_0\right),y_1,x\left)\right)$$ $$={\theta }^{\prime }(\lambda y_0{y}_1\alpha x.{\chi }_{\left(s\right(y_0),y_1)\in \alpha }g(s\left(y_0\right),y_1,x\left)\right)$$ And we can rewrite the endofunction a little bit $$={\theta }^{\prime }(\lambda y_0{y}_1\alpha x.{\chi }_{(s\times {\text{id}}_{{\Gamma }_1})(y_0,y_1)\in \alpha }g((s\times {\text{id}}_{{\Gamma }_1})(y_0,y_1),x\left)\right)$$ And finally apply our endofunction condition, since we’ve now got the function in a form that’s treating $y_0,y_1$ as part of the computational universe… $$\le \Theta (\lambda y_0{y}_{1}x.g(y_0,y_1,x\left)\right)$$ And we’re done, this establishes our desired result. $■$

Proposition 2.17: $\text{Br}\left(\Theta \right)$ is a continuous function of $\Theta$.

The way this proof will work is by describing a composition of functions that makes $\text{Br}\left(\Theta \right)$ from $\Theta$, and then showing that each of these functions is continuous, if ${\text{el}}^{\Gamma }\times \Phi$ is a finite set.

Claim: The bridge transform of some $\Theta$ is equal to (using ${\chi }_{{\text{el}}^{\Gamma }}$ to denote restricting an ultradistribution to the event $y\in \alpha$ and ${\chi }_{{\text{el}}^{\Gamma }}^{-1}$ to denote the inverse of said function, mapping an ultradistribution on ${\text{el}}^{\Gamma }$ to the largest ultradistribution that could have produced it via restriction) $${\chi }_{{\text{el}}^{\Gamma }}\left(\bigcap _{s:\Gamma \to \Gamma }{s}^{\ast }\right({\chi }_{{\text{el}}^{\Gamma }}^{-1}\left({\iota }_{\ast }\right(p{r}^{\ast }\left(\Theta \right)\left)\right)\left)\right)$$ Breaking down the unfamilar notation, the type of $pr$ is ${\text{el}}^{\Gamma }\times \Phi \to \Gamma \times \Phi$, just the usual projection. That asterisk up top is pullback along that function. The type of $\iota$ is ${\text{el}}^{\Gamma }\times \Phi \to \Gamma \times 2^{\Gamma }\times \Phi$. And $s^{\ast }$ is pullback along the function $\Gamma \times 2^{\Gamma }\times \Phi \to \Gamma \times 2^{\Gamma }\times \Phi$ given by $(s,{\text{id}}_{2^{\Gamma }},{\text{id}}_{\Phi })$.

Let’s unpack the exact conditions that cause a $\theta$ to lie in the set $${\chi }_{{\text{el}}^{\Gamma }}\left(\bigcap _{s:\Gamma \to \Gamma }{s}^{\ast }\right({\chi }_{{\text{el}}^{\Gamma }}^{-1}\left({\iota }_{\ast }\right(p{r}^{\ast }\left(\Theta \right)\left)\right)\left)\right)$$ First off, a $\theta$ is in this set iff it is supported over the event $y\in \alpha$, and it lies in the set $$\bigcap _{s:\Gamma \to \Gamma }{s}^{\ast }\left({\chi }_{{\text{el}}^{\Gamma }}^{-1}\right({\iota }_{\ast }\left(p{r}^{\ast }\right(\Theta \left)\right)\left)\right)$$ Which occurs iff $\theta$ is supported over the event $y\in \alpha$, and for all $s:\Gamma \to \Gamma$, $\theta$ lies in the set $$s^{\ast }\left({\chi }_{{\text{el}}^{\Gamma }}^{-1}\right({\iota }_{\ast }\left(p{r}^{\ast }\right(\Theta \left)\right)\left)\right)$$ Which occurs iff $\theta$ is suported over the event $y\in \alpha$, and for all $s:\Gamma \to \Gamma$, $s_{\ast }\left(\theta \right)$ lies in the set $${\chi }_{{\text{el}}^{\Gamma }}^{-1}\left({\iota }_{\ast }\right(p{r}^{\ast }\left(\Theta \right)\left)\right)$$ Which occurs iff $\theta$ is supported over the event $y\in \alpha$, and for all $s:\Gamma \to \Gamma$, ${\chi }_{{\text{el}}^{\Gamma }}\left(s_{\ast }\right(\theta \left)\right)$ lies in the set ${\iota }_{\ast }\left(p{r}^{\ast }\right(\Theta \left)\right)$

Now, $\iota$ is just doing a little bit of type conversion, so we’re justified in ignoring it. Anways, the previous thing occurs iff $\theta$ is supported over the event $y\in \alpha$, and for all $s:\Gamma \to \Gamma$, $p{r}_{\ast }\left({\chi }_{{\text{el}}^{\Gamma }}\right(s_{\ast }\left(\theta \right)\left)\right)\in \Theta$.

Which happens iff $\theta$ is supported over the event $y\in \alpha$ and for all $s:\Gamma \to \Gamma$ and $g:\Gamma \times \Phi \to [0,1]$, $$p{r}_{\ast }\left({\chi }_{{\text{el}}^{\Gamma }}\right(s_{\ast }\left(\theta \right)\left)\right)(\lambda yx.g(y,x\left)\right)\le \Theta (\lambda yx.g(y,x\left)\right)$$ However, unpacking the left-hand side, we get $$p{r}_{\ast }\left({\chi }_{{\text{el}}^{\Gamma }}\right(s_{\ast }\left(\theta \right)\left)\right)(\lambda yx.g(y,x\left)\right)$$ $$={\chi }_{{\text{el}}^{\Gamma }}\left(s_{\ast }\right(\theta \left)\right)(\lambda y\alpha x.g(y,x\left)\right)$$ $$=s_{\ast }\left(\theta \right)(\lambda y\alpha x.{\chi }_{y\in \alpha }g(y,x\left)\right)$$ $$=\theta (\lambda y\alpha x.{\chi }_{s\left(y\right)\in \alpha }g(s\left(y\right),x\left)\right)$$ Which is the exact condition for $\theta$ to lie in the bridge transform. So, we have an equivalence.

Now, since we’ve phrased the bridge transform as $${\chi }_{{\text{el}}^{\Gamma }}\left(\bigcap _{s:\Gamma \to \Gamma }{s}^{\ast }\right({\chi }_{{\text{el}}^{\Gamma }}^{-1}\left({\iota }_{\ast }\right(p{r}^{\ast }\left(\Theta \right)\left)\right)\left)\right)$$ We just need to establish that when all the sets are finite, then pullbacks are continuous, pushforwards are continuous, un-restrictions are continuous, intersections are continuous, and restrictions are continuous. Then, this would just be a particularly fancy continuous function, and accordingly, if ${\Theta }_n$ limited to $\Theta$, then $\text{Br}\left({\Theta }_n\right)$ would limit to $\text{Br}\left(\Theta \right)$.

Let’s establish that when the sets are finite, pullbacks are continuous. Let $g:X\to Y$, and $Y$ and $X$ be finite sets, and $\psi \in \square Y$. Then, we have $$g^{\ast }\left(\psi \right)(\lambda x.f(x\left)\right):=\psi (\lambda y.\underset{x\in g^{-1}\left(y\right)}{max}f(x\left)\right)$$ With the convention that maximizing over the empty set produces a value of 0. That is an alternate phrasing of pullback. We can then go $$\underset{n\to \infty }{lim}d\left(g^{\ast }\right({\psi }_n),g^{\ast }(\psi \left)\right)=\underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|g^{\ast }\left({\psi }_n\right)\left(f\right)-g^{\ast }\left(\psi \right)\left(f\right)|$$ $$=\underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|{\psi }_n(\lambda y.\underset{x\in g^{-1}\left(y\right)}{max}f(x\left)\right)-\psi (\lambda y.\underset{x\in g^{-1}\left(y\right)}{max}f(x\left)\right)|$$ $$\le \underset{n\to \infty }{lim}\underset{h:Y\to [0,1]}{sup}|{\psi }_n\left(h\right)-\psi \left(h\right)|=\underset{n\to \infty }{lim}d({\psi }_n,\psi )=0$$ Admittedly, this isn’t quite what our usual modified KR metric usually looks like. The reason we can do this is because we’re just dealing with functions in $[0,1]$, so the norm part of the modified KR metric doesn’t matter, and since our sets are finite, we can say that all points are distance 1 from each other, so all functions are 1-Lipschitz, and then the two metrics coincide. So, pullback along any function is continuous.

For pushforward, it’s easy because, if $\psi \in \square X$, then we’ve got $$\underset{n\to \infty }{lim}d\left(g_{\ast }\right({\psi }_n),g_{\ast }(\psi \left)\right)=\underset{n\to \infty }{lim}\underset{h:Y\to [0,1]}{sup}|g_{\ast }\left({\psi }_n\right)\left(h\right)-g_{\ast }\left(\psi \right)\left(h\right)|$$ $$=\underset{n\to \infty }{lim}\underset{h:Y\to [0,1]}{sup}|{\psi }_n(\lambda x.h(g\left(x\right)\left)\right)-\psi (\lambda x.h(g\left(x\right)\left)\right)|$$ $$\le \underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|{\psi }_n\left(f\right)-\psi \left(f\right)|=\underset{n\to \infty }{lim}d({\psi }_n,\psi )=0$$ For showing restrictions continuous, for the set $E\subseteq X$ that we’re updating on, $$\underset{n\to \infty }{lim}d\left({\chi }_E\right({\psi }_n),{\chi }_E(\psi \left)\right)=\underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|{\chi }_E\left({\psi }_n\right)\left(f\right)-{\chi }_E\left(\psi \right)\left(f\right)|$$ $$=\underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|{\psi }_n\left({\chi }_{x\in E}f\right(x\left)\right)-{\psi }_n\left({\chi }_{x\in E}f\right(x\left)\right)|$$ $$\le \underset{n\to \infty }{lim}\underset{f:X\to [0,1]}{sup}|{\psi }_n\left(f\right)-\psi \left(f\right)|=\underset{n\to \infty }{lim}d({\psi }_n,\psi )=0$$ For intersections… that will take a bit more work. We’ll have to use the equivalent formulation of closeness, that ${\psi }_n$ limits to $\psi$ iff the Hausdorff distance between the corresponding sets (according to the generalized KR measure) limits to 0. So, our task is to assume that ${\psi }_n$ limits to $\psi$, and ${\varphi }_n$ limits to $\varphi$, and show that ${\psi }_n\cap {\varphi }_n$ limits to $\psi \cap \varphi$. The bound we’ll manage to prove is that $$d({\psi }_n\cap {\varphi }_n,\psi \cap \varphi )\le |X|max\left(d\right({\psi }_n,\psi ),d({\varphi }_n,\varphi \left)\right)$$ Where $|X|$ is the number of elements in the finite set $X$. Here’s the basic argument. For any particular point in the set ${\psi }_n$, there’s a nearby point in $\psi$ (since the Hausdorff distance is low) with only $ϵ$ measure moved around or deleted. So, in particular, if all the measure moved or deleted was just deleted from ${\psi }_n$ instead, then that resulting contribution would be below the nearby contribution in $\psi$ that we picked, and so it would lie in $\psi$ as well due to downwards closure.

So, in particular, if ${\psi }_n$ and $\psi$ only have a Hausdorff distance of $ϵ$, then, taking any contribution in ${\psi }_n$ and subtracting $ϵ$ measure from \emph{all points} (if possible, if not, just remove measure till you’re at 0) is \emph{guaranteed} to make a point in $\psi$, and vice-versa.

And a corollary of that is that, given any contribution in ${\psi }_n\cap {\varphi }_n$, the “subtract $max\left(d\right({\psi }_n,\psi ),d({\varphi }_n,\varphi \left)\right)$ measure from each point” contribution is in $\psi$, also in $\varphi$, and at a maximum distance of $|X|max\left(d\right({\psi }_n,\psi ),d({\varphi }_n,\varphi \left)\right)$ from the original contribution. And this argument can be reversed to show that the limit of the intersections is the intersection of the limits (because hausdorff distance between the two goes to 0), so we do in fact have intersection being continuous.

And that just leaves un-restricting. Again, this will take a Hausdorff-distance argument. Fixing some contribution in ${\chi }_E^{-1}\left({\psi }_n\right)$, it can be broken down into an on-E part ${\theta }_{n,E}$, and an off-E part ${\theta }_{n,\neg E}$. When you restrict to E, then ${\theta }_{n,E}\in {\psi }_n$. Since ${\psi }_n$ is within $ϵ$ of $\psi$, there’s some ${\theta }_E\in \psi$ that’s within $ϵ$ of ${\theta }_{n,E}$. Then, let your point in ${\chi }_E^{-1}\left(\psi \right)$ be ${\theta }_E+{\theta }_{n,\neg E}$ (if there’s slightly more than 1 measure there, delete $ϵ$ measure from ${\theta }_{n,\neg E}$, or all the measure if there’s less than $ϵ$ present). It’s close to ${\theta }_{n,E}+{\theta }_{n,\neg E}$ because ${\theta }_{n,E}$ is close to ${\theta }_E$, the other component of it is unchanged, and maybe we deleted a little bit of excess measure which didn’t do much.

This line of argument shows that ${\psi }_n$ being close to the limit $\psi$ is sufficient to establish that the un-restriction of the two of them are comparably close together. So we have continuity for that, which is the last thing we needed.

Since we wrote the bridge transform as a sequence of continuous functions, we know it’s continuous (as long as all the involved sets are finite) $■$

Proposition 3.1: Let $X$ be a finite poset, $f:X\to R$ and $\Theta \in {\square }^{c}X$ downward closed. Define $f^{max}:X\to R$ by $f^{max}\left(x\right):={max}_{y\le x}f\left(y\right)$. Observe that $f^{max}$ is always non-decreasing. Then, $\Theta \left(f\right)=\Theta \left(f^{max}\right)$.

Proof: Pick a ${\theta }^{\prime }\in \Theta$ s.t. ${\theta }^{\prime }\left(f^{max}\right)=\Theta \left(f^{max}\right)$. Ie, a maximizing contribution. Let $k:X\to X$ be defined as $k:=\lambda x.{\text{argmax}}_{y\le x}f\left(y\right)$. Ie, it moves a point down to somewhere below it where it can attain the highest value according to $f$. Now, consider $k_{\ast }\left({\theta }^{\prime }\right)$. It’s present in $\Theta$ because $\Theta$ was, by assumption, downwards closed, and we just moved all the measure down.

Now, we have $$\Theta \left(f\right)=\underset{\theta \in \Theta }{max}\theta \left(f\right)\ge k_{\ast }\left({\theta }^{\prime }\right)\left(f\right)={\theta }^{\prime }(\lambda x.f(k\left(x\right)\left)\right)={\theta }^{\prime }(\lambda x.f({\text{argmax}}_{y\le x}f\left(y\right)\left)\right)$$ $$={\theta }^{\prime }(\lambda x.\underset{y\le x}{max}f(y\left)\right)={\theta }^{\prime }\left(f^{max}\right)=\Theta \left(f^{max}\right)\ge \Theta \left(f\right)$$ And so, all inequalities must be equalities, proving that $\Theta \left(f^{max}\right)\ge \Theta \left(f\right)$. In order, the connectives were: unpacking definitions, using downward closure to conclude that $k_{\ast }\left({\theta }^{\prime }\right)\in \Theta$, unpacking pushforwards, unpacking the definition of $k$, using that applying a function to the argmax of inputs to that function just makes the max of the function, folding the definition of $f^{max}$ back up, using that ${\theta }^{\prime }$ was selected to maximize $f^{max}$, and applying monotonicity. Done! $■$

Proposition 4.1: Consider some $\Gamma$, $\Phi$, a relation $Q\subseteq \Gamma \times \Phi$ and a PUCK $\Xi$ over $Q$. Let $\Theta :={\top }_{\Gamma }⋉\Xi$. Then, $$\text{Br}\left(\Theta \right)=[{\top }_{\Gamma }⋉({\text{sus}}^{\Theta }⋊\Xi ){]}^{\downarrow }=[{\top }_{\Gamma }⋉(Q^{-1}⋊\Xi ){]}^{\downarrow }$$

First off, I’m not terribly picky about variable ordering, so I’ll just write our desired proof target as $$\text{Br}\left(\Theta \right)=[{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }{]}^{\downarrow }=[{\top }_{\Gamma }⋉\Xi ⋉Q^{-1}{]}^{\downarrow }$$ The way we’ll do this is by establishing the following result. For all monotone $f^{\prime }:{\text{el}}^{\Gamma }\times \Phi \to [0,1]$, we have $$\text{Br}\left(\Theta \right)\left(f^{\prime }\right)\le [{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }]\left(f^{\prime }\right)\le [{\top }_{\Gamma }⋉\Xi ⋉Q^{-1}]\left(f^{\prime }\right)\le \text{Br}\left(\Theta \right)\left(f^{\prime }\right)$$ Why does that suffice? Well, assume hypothetically that the result held. Since the inequalities go in a circle, we have equality for all monotone functions. And then, for some non-monotone function $f$, we can go $$\text{Br}\left(\Theta \right)\left(f\right)=\text{Br}\left(\Theta \right)\left(f^{max}\right)=[{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }]\left(f^{max}\right)$$ $$=[{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }{]}^{\downarrow }(f^{max})=[{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }{]}^{\downarrow }\left(f\right)$$ and swap out ${\text{sus}}^{\Theta }$ for $Q^{-1}$ to show the other equality, and then we’d have equality of the three ultradistributions on all functions, so they’re equal.

For the equalities in the above equation, the first one arose because of Proposition 2.4 (bridge transforms are always downwards closed) and Proposition 3.1 (downwards-closed things let you swap out $f$ for $f^{max}$ and it doesn’t affect the value). The second equality arose because $f^{max}$ is a monotone function and by assumption, we have equality for monotone functions. The third equality would arise because taking the downwards closure doesn’t affect the expectation value of monotone functions. If you add a bunch of contributions made by measure flowing down, that’s just strictly worse from the perspective of a monotone function and doesn’t change expectation value. And the fourth equality arises from Proposition 3.1 again.

So, we just need to prove the following three inequalities, for monotone functions $f$. $$\text{Br}\left(\Theta \right)\left(f\right)\le [{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }]\left)\right(f)\le [{\top }_{\Gamma }⋉\Xi ⋉Q^{-1}\left]\right(f)\le \text{Br}(\Theta \left)\right(f)$$ The first one is easily addressable by Proposition 2.7. By proposition 2.7 and the definition of $\Theta$, we have $$\text{Br}\left(\Theta \right)\subseteq (\Theta ⋉{\text{sus}}^{\Theta }{)}^{\downarrow }=[{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }{]}^{\downarrow }$$ And so, for monotone functions $f$, we have $$\text{Br}\left(\Theta \right)\left(f\right)\le [{\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta }]\left)\right(f)$$ Done.

Now to show our second inequality. $$({\top }_{\Gamma }⋉\Xi ⋉{\text{sus}}^{\Theta })(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.{\delta }_{{\text{sus}}^{\Theta }\left(x\right)}(\lambda \alpha .f(y,x,\alpha )\left)\right)$$ $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,{\text{sus}}^{\Theta }\left(x\right)\left)\right)$$ Unpack the definition of the set $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\left\{y^{\prime }|\right(y^{\prime },x)\in \text{supp}\Theta \}\left)\right)$$ Unpack the definition of $\Theta$ $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\left\{y^{\prime }|\right(y^{\prime },x)\in \text{supp}{\top }_{\Gamma }⋉\Xi \}\left)\right)$$ The condition $(y^{\prime },x)\in \text{supp}{\top }_{\Gamma }⋉\Xi$ is equivalent to $x\in \text{supp}\Xi \left(y^{\prime }\right)$. After all, if $x\in \text{supp}\Xi \left(y^{\prime }\right)$, the distribution ${\delta }_{y^{\prime }}$ lies in ${\top }_{\Gamma }$, so ${\delta }_{y^{\prime }}⋉\Xi$ would certify that $(y^{\prime },x)\in \text{supp}{\top }_{\Gamma }⋉\Xi$. And if $x\notin \text{supp}\Xi \left(y^{\prime }\right)$, then no matter the distribution in ${\top }_{\Gamma }$ or kernel selected from $\Xi$, if $y^{\prime }$ gets picked, then the kernel selected from $\Xi$ isn’t going to be making $x$ along with it. Since we have that iff characterization, we have $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\{y^{\prime }|x\in \text{supp}\Xi (y^{\prime }\left)\right\}\left)\right)$$ $\Xi \left(y^{\prime }\right)$ is the union of a bunch of $k\left(y^{\prime }\right)$ for $k\in \Pi$ (and convex hull), so its support is equal to the union of the supports for the $k\left(y^{\prime }\right)$. $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\{y^{\prime }|x\in \bigcup _{k\in \Pi }\text{supp}k(y^{\prime }\left)\right\}\left)\right)$$ Then, since each $k$ is a PoCK over $Q$, $k\left(y^{\prime }\right)$ is the restriction of some measure ${\varpi }^k$ to the set $Q\left(y\right)$, which will be written as ${\chi }_{Q\left(y^{\prime }\right)}{\varpi }^k$. $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\{y^{\prime }|x\in \bigcup _{k\in \Pi }\text{supp}({\chi }_{Q\left(y^{\prime }\right)}{\varpi }^k\left)\right\}\left)\right)$$ And now we’re about to get an inequality.$f$ is monotone, so making the associated set bigger (easier to fulfill the defining condition) should always increase the value of $f$, and by monotonicity, increase the expectation value, so we get $$\le ({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\{y^{\prime }|x\in Q(y^{\prime }\left)\right\}\left)\right)$$ Then restate $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,\left\{y^{\prime }|\right(x,y^{\prime })\in Q\}\left)\right)$$ $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.f(y,x,Q^{-1}\left(x\right)\left)\right)$$ And pack back up as a semidirect product. $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y,x,\alpha )\left)\right)$$ $$=({\top }_{\Gamma }⋉\Xi ⋉Q^{-1})(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ And we have our second $\le$ inequality established!

Now, onto the third inequality. $$({\top }_{\Gamma }⋉\Xi ⋉Q^{-1})(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ Unpack the semidirect products $$={\top }_{\Gamma }(\lambda y.\Xi (y\left)\right(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y,x,\alpha \left)\right)\left)\right)$$ And what top means $$=\underset{y\in \Gamma }{max}\Xi \left(y\right)(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y,x,\alpha )\left)\right)$$ And as for $\Xi$… well, each $\Xi \left(y\right)$ is the convex hull of the various $k\left(y\right)$, for $k\in \Pi$. So, the expectation for $\Xi \left(y\right)$ is the maximum expectation for the various $k\left(y\right)$, so we can rewrite as $$=\underset{y\in \Gamma }{max}\underset{k\in \Pi }{max}k\left(y\right)(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y,x,\alpha )\left)\right)$$ Pick a particular $y^{\ast }$ and $k^{\ast }$ that attain the maximal value $$=k^{\ast }\left(y^{\ast }\right)(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y^{\ast },x,\alpha )\left)\right)$$ Reexpress a little bit $$={\delta }_{y^{\ast }}(\lambda y.k^{\ast }(y\left)\right(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .f(y,x,\alpha \left)\right))$$ And pack this back up as a semidirect product $$=({\delta }_{y^{\ast }}⋉k^{\ast }⋉Q^{-1})(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ And then we’ll be showing that this contribution lies in $\text{Br}\left(\Theta \right)$. Once we’ve done that, we can go $$\le \underset{{\theta }^{\prime }\in \text{Br}\left(\Theta \right)}{max}{\theta }^{\prime }(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ $$=\text{Br}\left(\Theta \right)(\lambda y\alpha x.f(y,x,\alpha \left)\right)$$ And we’d be done, having proven the third inequality and the last one to finish up the proof. So, now our proof target switches to showing that $({\delta }_{y^{\ast }}⋉k^{\ast }⋉Q^{-1})\in \text{Br}\left(\Theta \right)$. We can show this if we show the support condition and the endofunction condition.

For the support condition, we have $$({\delta }_{y^{\ast }}⋉k^{\ast }⋉Q^{-1})(\lambda y\alpha x.{\chi }_{y\notin \alpha })$$ $$={\delta }_{y^{\ast }}(\lambda y.k^{\ast }(y\left)\right(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .{\chi }_{y\notin \alpha })\left)\right)$$ $$={\delta }_{y^{\ast }}(\lambda y.k^{\ast }(y\left)\right(\lambda x.{\chi }_{y\notin Q^{-1}\left(x\right)}\left)\right)$$ $$=k^{\ast }\left(y^{\ast }\right)(\lambda x.{\chi }_{y^{\ast }\notin Q^{-1}\left(x\right)})$$ And then we use that the $k^{\ast }\left(y^{\ast }\right)$ are all of the form “take this measure, restrict it to $Q\left(y^{\ast }\right)$”, to get $$=\left({\chi }_{Q\left(y^{\ast }\right)}{\varpi }^{k^{\ast }}\right)(\lambda x.{\chi }_{y^{\ast }\notin Q^{-1}\left(x\right)})$$ $$={\varpi }^{k^{\ast }}(\lambda x.{\chi }_{x\in Q\left(y^{\ast }\right)}{\chi }_{y^{\ast }\notin Q^{-1}\left(x\right)})$$ Unpacking the definitions, we get $$={\varpi }^{k^{\ast }}(\lambda x.{\chi }_{(x,y^{\ast })\in Q}{\chi }_{(x,y^{\ast })\notin Q})=0$$ And so, this contribution is indeed supported on $(y,\alpha )$ pairs s.t. $y\in \alpha$.

Now for the endofunction condition. As usual, fix an $s$ and a $g$. $$({\delta }_{y^{\ast }}⋉k^{\ast }⋉Q^{-1})(\lambda y\alpha x.{\chi }_{s\left(y\right)\in \alpha }g(s\left(y\right),x\left)\right)$$ Unpack the semidirect product $$={\delta }_{y^{\ast }}(\lambda y.k^{\ast }(y\left)\right(\lambda x.{\delta }_{Q^{-1}\left(x\right)}(\lambda \alpha .{\chi }_{s\left(y\right)\in \alpha }g(s\left(y\right),x\left)\right)\left)\right)$$ Plug in the dirac-deltas $$=k^{\ast }\left(y^{\ast }\right)(\lambda x.{\chi }_{s\left(y^{\ast }\right)\in Q^{-1}\left(x\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ Reexpress the set membership criterion a bit $$=k^{\ast }\left(y^{\ast }\right)(\lambda x.{\chi }_{x\in Q\left(s\right(y^{\ast }\left)\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ And the contribution at the start $$=\left({\chi }_{Q\left(y^{\ast }\right)}{\varpi }^{k^{\ast }}\right)(\lambda x.{\chi }_{x\in Q\left(s\right(y^{\ast }\left)\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ Distribute it in as an indicator function. $$={\varpi }^{k^{\ast }}(\lambda x.{\chi }_{x\in Q\left(y^{\ast }\right)}{\chi }_{x\in Q\left(s\right(y^{\ast }\left)\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ Pull the other indicator function out. $$=\left({\chi }_{Q\left(s\right(y^{\ast }\left)\right)}{\varpi }^{k^{\ast }}\right)(\lambda x.{\chi }_{x\in Q\left(y^{\ast }\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ Rewrite with $k^{\ast }$ $$=k^{\ast }\left(s\right(y^{\ast }\left)\right)(\lambda x.{\chi }_{x\in Q\left(y^{\ast }\right)}g(s\left(y^{\ast }\right),x\left)\right)$$ Use an inequality to get rid of the indicator function $$\le k^{\ast }\left(s\right(y^{\ast }\left)\right)(\lambda x.g(s\left(y^{\ast }\right),x\left)\right)$$ Rewrite it a bit $$={\delta }_{s\left(y^{\ast }\right)}(\lambda y.k^{\ast }(y\left)\right(\lambda x.g(y,x)\left)\right)$$ Swap out $k^{\ast }\left(y\right)$ for $\Xi \left(y\right)$, the latter is larger $$\le {\delta }_{s\left(y^{\ast }\right)}(\lambda y.\Xi (y\left)\right(\lambda x.g(y,x)\left)\right)$$ Swap out ${\delta }_{s\left(y^{\ast }\right)}$ for ${\top }_{\Gamma }$, the latter is larger $$\le {\top }_{\Gamma }(\lambda y.\Xi (y\left)\right(\lambda x.g(y,x)\left)\right)$$ $$=({\top }_{\Gamma }⋉\Xi )(\lambda yx.g(y,x\left)\right)$$ Abbreviate $$=\Theta (\lambda yx.g(y,x\left)\right)$$ And bam, endofunction condition is shown, the entire proof goes through now. $■$

Corollary 4.3: Suppose that for any $d\in D$ and $\pi :H\to A$ s.t. $d\in \text{supp}W\left(\pi \right)$, it holds that $dC\pi$. That is, the observations $W$ predicts to receive from the computer are consistent with the chosen policy. Let $L:D\to R$ be a Cartesian loss function and $\pi :H\to A$ a policy. Then, $$\left({\text{pr}}_{{\text{el}}^{\Gamma }}\text{Br}\right({\Theta }_W)\cap {\text{C}}_{\text{fair}}^{\pi })\left(L^{\text{phys}}\right)=W(\pi ;L)$$

I’m going to be proceeding very cautiously here. First off, make our $\pi$ value visually distinct by swapping it out for ${\pi }^{\ast }$ $$\left({\text{pr}}_{{\text{el}}^{\Gamma }}\text{Br}\right({\Theta }_W)\cap {\text{C}}_{\text{fair}}^{{\pi }^{\ast }})\left(L^{\text{phys}}\right)$$ Now, by the identifications we made earlier, we can identify $\Gamma$ with $A^H$, the space of policies. Using that to unpack the function a little bit, we have $$=\left({\text{pr}}_{{\text{el}}^{\Gamma }}\text{Br}\right({\Theta }_W)\cap {\text{C}}_{\text{fair}}^{{\pi }^{\ast }})(\lambda \pi \alpha .L^{\text{phys}}(\pi ,\alpha \left)\right)$$ Now, we note that intersecting with top of a particular set is equivalent to updating on the indicator function for that set. Using definition 1.5 to unpack ${\text{C}}_{\text{fair}}^{{\pi }^{\ast }}$, we get $$=\left({\text{pr}}_{{\text{el}}^{\Gamma }}\text{Br}\right({\Theta }_W\left)\right)(\lambda \pi \alpha .{\chi }_{\forall h\in H_{\pi ,\alpha }:G^{\pi }\left(h\right)={\pi }^{\ast }\left(h\right)}{L}^{\text{phys}}(\pi ,\alpha \left)\right)$$ Apply that $G^{\pi }\left(h\right)$ is “what would the agent do on $h$ if the agent is copying the behavior of $\pi$”, so we can rephrase as: $$=\left({\text{pr}}_{{\text{el}}^{\Gamma }}\text{Br}\right({\Theta }_W\left)\right)(\lambda \pi \alpha .{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}{L}^{\text{phys}}(\pi ,\alpha \left)\right)$$ Pull off the projection, and use $d$ for a destiny in $D$. $$=\text{Br}\left({\Theta }_W\right)(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}{L}^{\text{phys}}(\pi ,\alpha \left)\right)$$ At this point, we use that ${\Theta }_W:={\top }_{\Gamma }⋉W$, and that $W$ is a PUCK over $Q_0$ and Proposition 4.1 to go $$=[{\top }_{\Gamma }⋉W⋉Q_0^{-1}{]}^{\downarrow }(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}{L}^{\text{phys}}(\pi ,\alpha ))$$ Before we can remove the downwards closure, we’ll want to verify the function is monotone. So, we’ll want to start unpacking the physicalist loss next. Applying definition 3.1, and using $d^{\prime }$ instead of $g$ to remember it’s a destiny, we have $$=[{\top }_{\Gamma }⋉W⋉Q_0^{-1}{]}^{\downarrow }(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}\underset{ha:ha\in X_{\pi ,\alpha }}{min}\underset{d^{\prime }:ha⊑d^{\prime }}{max}L\left(d^{\prime }\right))$$ Next up is unpacking $X_{\pi ,\alpha }$. Using definition 3.1, it’s $$=[{\top }_{\Gamma }⋉W⋉Q_0^{-1}{]}^{\downarrow }(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,\alpha }\times A\wedge (\forall {\pi }^{\prime }\in \alpha :G^{{\pi }^{\prime }}(h)=a)}{min}\underset{d^{\prime \prime }:ha⊑d^{\prime }}{max}L\left(d^{\prime }\right))$$ At this point, we can, again, treat $G^{{\pi }^{\prime }}\left(h\right)$ the same as ${\pi }^{\prime }\left(h\right)$. $$=[{\top }_{\Gamma }⋉W⋉Q_0^{-1}{]}^{\downarrow }(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,\alpha }\times A\wedge (\forall {\pi }^{\prime }\in \alpha :{\pi }^{\prime }(h)=a)}{min}\underset{d^{\prime }:ha⊑d^{\prime \prime }}{max}L\left(d^{\prime }\right))$$ And now we need to take a moment to show that $H_{\pi ,\alpha }$ gets smaller when $\alpha$ gets larger. Applying definition 1.5, the event $h\in H_{\pi ,\alpha }$ unpacks as $$(\forall h^{\prime }a⊏h,{\pi }^{\prime }\in \alpha :G^{{\pi }^{\prime }}(h^{\prime })=a)\wedge (\exists d^{\prime }:h⊏d^{\prime }\wedge d^{\prime }C\pi )$$ Now, if $\alpha$ becomes a larger set, then it gets harder for the first condition to be fulfilled, so the set $H_{\pi ,\alpha }$ shrinks. Now, since this happens, it means that if $\alpha$ gets bigger, it gets more difficult for the prerequisite of the implication in the indicator function to be fulfilled, so the implication is more likely to hold. Further, the minimization is taking place over a smaller set, so the loss goes up. So our function is monotone in $\alpha$, and we can remove the downwards closure. $$=({\top }_{\Gamma }⋉W⋉Q_0^{-1})(\lambda \pi \alpha d.{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,\alpha }\times A\wedge (\forall {\pi }^{\prime }\in \alpha :{\pi }^{\prime }(h)=a)}{min}\underset{d^{\prime }:ha⊑d^{\prime \prime }}{max}L\left(d^{\prime }\right))$$ Unpacking the semidirect product, it is $$={\top }_{\Gamma }(\lambda \pi .W(\pi \left)\right(\lambda d.{\delta }_{Q_0^{-1}\left(d\right)}(\lambda \alpha .{\chi }_{\forall h\in H_{\pi ,\alpha }:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,\alpha }\times A\wedge (\forall {\pi }^{\prime }\in \alpha :{\pi }^{\prime }(h)=a)}{min}\underset{d^{\prime }:ha⊑d^{\prime }}{max}L\left(d^{\prime }\right)\left)\right))$$ Substituting in the dirac-delta everywhere that $\alpha$ is, we get $$={\top }_{\Gamma }(\lambda \pi .W(\pi \left)\right(\lambda d.{\chi }_{\forall h\in H_{\pi ,Q_0^{-1}\left(d\right)}:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,Q_0^{-1}\left(d\right)}\times A\wedge (\forall {\pi }^{\prime }\in Q_0^{-1}(d):{\pi }^{\prime }(h)=a)}{min}\underset{d^{\prime }:ha⊑d^{\prime }}{max}L\left(d^{\prime }\right)\left)\right)$$ Now, $Q_0^{-1}\left(d\right)$ is the set of policies ${\pi }^{\prime }$ s.t. ${\pi }^{\prime }{Q}_{0}d$. The “this policy is consistent with this destiny” relation. Also let’s swap out ${\top }_{\Gamma }$ for maximization $$=\underset{\pi }{max}W\left(\pi \right)(\lambda d.{\chi }_{\forall h\in H_{\pi ,Q_0^{-1}\left(d\right)}:\pi \left(h\right)={\pi }^{\ast }\left(h\right)}$$ $$\underset{ha:ha\in H_{\pi ,Q_0^{-1}\left(d\right)}\times A\wedge (\forall {\pi }^{\prime }{Q}_{0}d:{\pi }^{\prime }(h)=a)}{min}\underset{d^{\prime }:ha⊑d^{\prime }}{max}L\left(d^{\prime }\right))$$ Now, we’re going to try to address that minimum, and show that the only $ha$ that fulfill the conditions are exactly those $ha⊑d$. This requires showing that $ha⊑d$ is a sufficient condition to fulfill the relevant properties, and then to show that $ha⋢d$ implies a failure of one of the properties.

So, first up. Assume $ha⊑d$. Then, for any ${\pi }^{\prime }$, $d{Q}_0{\pi }^{\prime }$ and $ha⊑d$ \emph{must} imply that ${\pi }^{\prime }\left(h\right)=a$, that’s what policy consistency means. Also, $h\in H_{\pi ,Q_0^{-1}\left(d\right)}$ unpacks as the two conditions $$\forall h^{\prime }{a}^{\prime },{\pi }^{\prime }:h^{\prime }{a}^{\prime }⊏h\wedge d{Q}_0{\pi }^{\prime }\to {\pi }^{\prime }\left(h^{\prime }\right)=a^{\prime }$$ $$\exists d^{\prime }:h⊏d^{\prime }\wedge d^{\prime }C\pi$$ As for the first condition,clearly, if ${\pi }^{\prime }$ is consistent with $d$, it’s consistent with $ha$ because $ha⊑d$, and so it must be consistent with any prefix of $ha$, so the first condition holds.

For the second condition, $d$ is a valid choice, because we assumed $ha⊑d$, and $dC\pi$ occurs always, because $W\left(\pi \right)$ always being supported on $d$ s.t.$dC\pi$ was one of our problem assumptions.

So, we have one implication direction down. Now for the reverse implication direction. Assume $ha⋢d$. Then there are two possibilities. the first possibility is that $ha$ first diverges from $d$ on an observation. The second possibility is that $ha$ first diverges from $d$ on an action.

For the first possibility, it’s possible to make two policies which are consistent with $d$ but also differ in their actions on history $h$, because $h$ isn’t a prefix of $d$ if $ha$ first differs from $d$ on an observation.

For the second possibility, it’s ruled out by either the condition for $h\in H_{\pi ,Q_0^{-1}\left(d\right)}$ that goes $$\forall h^{\prime }{a}^{\prime },{\pi }^{\prime }:h^{\prime }{a}^{\prime }⊏h\wedge {\pi }^{\prime }{Q}_{0}d\to {\pi }^{\prime }\left(h^{\prime }\right)=a^{\prime }$$ or the extra condition that $$\forall {\pi }^{\prime }:{\pi }^{\prime }{Q}_{0}d\to {\pi }^{\prime }\left(h\right)=a$$ applied to the first a-history prefix that deviates from $d$, because ${\pi }^{\prime }{Q}_{0}d$ implies that ${\pi }^{\prime }$