LBIT Proofs 2: Propositions 10-18

[Diffractor]

Proposition 10: Mixture, updating, and continuous pushforward preserve the properties indicated by the diagram, and always produce an infradistribution.

We’ll start with showing that mixture, updating, and continuous pushfoward are always infradistributions, and then turn to property verification.

We know from the last post that mixture, updating, and continuous pushfoward preserve all infradistribution properties (although you need to be careful about whether mixture preserves Lipschitzness, you need that the expected value of the Lipschitz constant is finite), but we added the new one about compact almost-support, so that’s the only part we need to re-verify.

To show that mixture has compact almost-support, remember that
$\left(E_{\zeta }{h}_i\right)\left(f\right)=E_{\zeta }\left(h_i\right(f\left)\right)$
Now, fix an $ϵ$, we will craft a compact set that accounts for all but $ϵ$ of why functions have the expectation values they do. There is some n where ${\sum }_{i>n}{\zeta }_i{\lambda }_i^{\odot }<\frac{ϵ}{2}$, where ${\lambda }_i^{\odot }$ is the Lipschitz constant of the infradistribution $h_i$. Then, let $C_{ϵ}$ be ${\bigcup }_{i\le n}{C}_{i,\frac{ϵ}{2}}$, the union of the compact $\frac{ϵ}{2}$-almost-supports for the infradistributions $h_i,i\le n$. This is a finite union of compact sets, so it’s compact.

Now we can go:
$|\left(E_{\zeta }{h}_i\right)\left(f\right)-\left(E_{\zeta }{h}_i\right)\left(f\right)|=|E_{\zeta }\left(h_i\right(f\left)\right)-E_{\zeta }\left(h_i\right(f^{\prime }\left)\right)|\le {\sum }_i{\zeta }_i|h_i\left(f\right)-h_i\left(f^{\prime }\right)|$
$={\sum }_{i\le n}{\zeta }_i|h_i\left(f\right)-h_i\left(f^{\prime }\right)|+{\sum }_{i>n}{\zeta }_i|h_i\left(f\right)-h_i\left(f^{\prime }\right)|\le {\sum }_{i\le n}{\zeta }_i\frac{ϵ}{2}d(f,f^{\prime })+{\sum }_{i>n}{\zeta }_i{\lambda }_i^{\odot }d(f,f^{\prime })$
$=d(f,f^{\prime })(\frac{ϵ}{2}{\sum }_{i\le n}{\zeta }_i+{\sum }_{i>n}{\zeta }_i{\lambda }_i^{\odot })<d(f,f^{\prime })(\frac{ϵ}{2}+\frac{ϵ}{2})=ϵd(f,f^{\prime })$
The first equality is reexpressing mixtures, and the first inequality is moving the expectation outside the absolute value which doesn’t decrease value, then we break up the expectation for the second equality. The second inequality is because the gap between $h_i\left(f\right)$ and $h_i\left(f^{\prime }\right)$ has a trivial upper bound from the Lipschitzness of $h_i$, and for $i\le n$, we have that $f$ and $f^{\prime }$ agree on the union of the $\frac{ϵ}{2}$-almost-supports for the $h_i,i\le n$, so a particular infradistribution, by the definition of an almost-support, has these two expectations having not-very-different values. Then we just pull the gap between $f$ and $f^{\prime }$ out, and use the fact that for the mixture to work, ${\sum }_i{\zeta }_i{\lambda }_i^{\odot }<\infty$, and we picked n big enough for that last tail of the infinite sum to be small. Then we’re done.

Now, we will show compact almost-support for $h{|}^{g}L$ assuming $h$ has compact almost-support. Fix an $ϵ$. Your relevant set for $\text{support}\left(L\right)$ will be
$C_{\frac{ϵP_h^g\left(L\right)}{2}}\cap \left\{x|L\right(x)\ge \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}\}$
Where the first term is a compact set that is a $\frac{ϵP_h^g\left(L\right)}{2}$-almost-support for $h$, and that last set is a sort of “this point must be likely enough”. ${\lambda }^{\odot }$ will be the Lipschitz constant of the original $h$. Yes, this intersection may be empty.

Now, here’s how things go. Let $f$ and $f^{\prime }$ agree on that intersection. (if it’s the empty set, then it can be any two functions). We can go:
$|\left(h{|}^{g}L\right)\left(f\right)-\left(h{|}^{g}L\right)\left(f^{\prime }\right)|=|\frac{h\left(f{★}^{L}g\right)-h\left(0{★}^{L}g\right)}{h\left(1{★}^{L}g\right)-h\left(0{★}^{L}g\right)}-\frac{h\left(f^{\prime }{★}^{L}g\right)-h\left(0{★}^{L}g\right)}{h\left(1{★}^{L}g\right)-h\left(0{★}^{L}g\right)}|$
$=\frac{1}{h\left(1{★}^{L}g\right)-h\left(0{★}^{L}g\right)}|h\left(f{★}^{L}g\right)-h\left(0{★}^{L}g\right)-h\left(f^{\prime }{★}^{L}g\right)+h\left(0{★}^{L}g\right)|$
$=\frac{1}{P_h^g\left(L\right)}|h\left(f{★}^{L}g\right)-h\left(f^{\prime }{★}^{L}g\right)|=\frac{1}{P_h^g\left(L\right)}|h(Lf+(1-L\left)g\right)-h(L{f}^{\prime }+(1-L\left)g\right)|$
So far, this is just a standard sequence of rewrites. The definition of the update, pulling the fraction out, using $P_h^g\left(L\right)$ to abbreviate the rescaling term, and unpacking what ${★}^L$ means.

Now, let’s see how different $Lf+(1-L)g$ and $L{f}^{\prime }+(1-L)g$ are on the set $C_{\frac{ϵP_h^g\left(L\right)}{2}}$. One of two things will occur. Our first possibility is that an $x$ in that compact set also has $L\left(x\right)\ge \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}$. Then
$x\in C_{\frac{ϵP_h^g\left(L\right)}{2}}\cap \left\{x|L\right(x)\ge \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}\}$
and $f,f^{\prime }$ were selected to be equal on that set, so the two functions will be identical on that point. Our second possibility is that $x$ in that compact set will have $L\left(x\right)<\frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}$. In that case,
$|L\left(x\right)f\left(x\right)+(1-L(x\left)\right)g\left(x\right)-L\left(x\right)f^{\prime }\left(x\right)-(1-L(x\left)\right)g\left(x\right)|$
$=|L\left(x\right)f\left(x\right)-L\left(x\right)f^{\prime }\left(x\right)|=L\left(x\right)|f\left(x\right)-f^{\prime }\left(x\right)|\le \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}d(f,f^{\prime })$
Because $L\left(x\right)<\delta$.

Putting this together, $Lf+(1-L)g$ and $L{f}^{\prime }+(1-L)g$ are only $\frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}d(f,f^{\prime })$ apart when restricted to the compact set $C_{\frac{ϵP_h^g\left(L\right)}{2}}$. By Lemma 2, we can then show that
$|h(Lf+(1-L\left)g\right)-h(L{f}^{\prime }+(1-L\left)g\right)|\le {\lambda }^{\odot }\cdot \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}d(f,f^{\prime })+\frac{ϵP_h^g\left(L\right)}{2}d(Lf+(1-L)g,L{f}^{\prime }+(1-L\left)g\right)$
And, we also know that:
$d(Lf+(1-L)g,L{f}^{\prime }+(1-L\left)g\right)=d(Lf,L{f}^{\prime })\le d(f,f^{\prime })$
Because $L\in [0,1]$. Making that substitution, we have:
$|h(Lf+(1-L\left)g\right)-h(L{f}^{\prime }+(1-L\left)g\right)|\le {\lambda }^{\odot }\cdot \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}d(f,f^{\prime })+\frac{ϵP_h^g\left(L\right)}{2}d(f,f^{\prime })$
$=\frac{ϵP_h^g\left(L\right)}{2}d(f,f^{\prime })+\frac{ϵP_h^g\left(L\right)}{2}d(f,f^{\prime })=ϵP_h^g\left(L\right)d(f,f^{\prime })$

Backing up to earlier, we had established that
$|\left(h{|}^{g}L\right)\left(f\right)-\left(h{|}^{g}L\right)\left(f^{\prime }\right)|=\frac{1}{P_h^g\left(L\right)}|h(Lf+(1-L\left)g\right)-h(L{f}^{\prime }+(1-L\left)g\right)|$
and from shortly above, we established that
$|h(Lf+(1-L\left)g\right)-h(L{f}^{\prime }+(1-L\left)g\right)|\le ϵP_h^g\left(L\right)d(f,f^{\prime })$
Putting these together,
$|\left(h{|}^{g}L\right)\left(f\right)-\left(h{|}^{g}L\right)\left(f^{\prime }\right)|<ϵd(f,f^{\prime })$
For any two functions $f$ and $f^{\prime }$ which agree on
$C_{\frac{ϵP_h^g\left(L\right)}{2}}\cap \left\{x|L\right(x)\ge \frac{ϵP_h^g\left(L\right)}{2{\lambda }^{\odot }}\}$
Witnessing that said set is an $ϵ$-almost-support for $h{|}^{g}L$.

All we need to finish up is to show that this is a compact set in $\text{support}\left(L\right)$ equipped with the subspace topology. This can be done by observing that in the original space $X$ it’s a compact set, due to being the intersection of a compact set and a closed set. In the subspace topology, if we try to make an open cover of it, all the open sets that cover it in the subspace topology are the restrictions of open sets in the original topology, so we have an open cover of this set in the original topology, and we can make a finite subcover, so it’s compact in the subspace topology as well.

Thus, for any $ϵ$, we can make a compact (in $\text{support}\left(L\right)$) $ϵ$-almost-support for $h{|}^{g}L$, so $h{|}^{g}L$ has compact almost-support and we’ve verified the last condition for an update of an infradistribution to be an update.

Now for deterministic pushfoward. Fix an $ϵ$, and let your appropriate set for $g_{\ast }\left(h\right)$ be $g\left(C_{ϵ}\right)$ where $C_{ϵ}$ is a compact $ϵ$-almost-support for $h$. The image of a compact set is compact, so that part is taken care of. We still need to check that it’s an $ϵ$-almost-support for $g_{\ast }\left(h\right)$. Let $f,f^{\prime }$ be equal on this set. Then
$|g_{\ast }\left(h\right)\left(f\right)-g_{\ast }\left(h\right)\left(f^{\prime }\right)|=|h(f\circ g)-h(f^{\prime }\circ g)|\le ϵd(f\circ g,f^{\prime }\circ g)$
$=ϵ{sup}_x|f\left(g\right(x\left)\right),f^{\prime }\left(g\right(x\left)\right)|\le ϵ{sup}_y|f\left(y\right),f^{\prime }\left(y\right)|=ϵd(f,f^{\prime })$
And we’re done. This is because, for any point $x\in C_{ϵ}$, feeding it through $g$ makes a point in $g\left(C_{ϵ}\right)$, and feeding it through $f$ and $f^{\prime }$ produces identical results because they agree on $g\left(C_{ϵ}\right)$. Therefore, $f\circ g$ and $f^{\prime }\circ g$ agree on $C_{ϵ}$ and thus can have values only $ϵd(f\circ g,f^{\prime }\circ g)$ apart, which is actually upper-bounded by $ϵd(f,f^{\prime })$. $g\left(C_{ϵ}\right)$ is thus a compact $ϵ$-almost-support for $g_{\ast }\left(h\right)$, and this can be done for any $ϵ$, so $g_{\ast }\left(h\right)$ has compact almost-support.

Since these three operations always produce infradistributions (as we’ve shown, we verified the last condition). Updating only has two properties to check, preserving homogenity when $g=0$ and cohomogenity when $g=1$, so let’s get that knocked out.

Homogenity using homogenity for h
$\left(h{|}^{0}L\right)\left(af\right)=\frac{h\left(af{★}^{L}0\right)-h\left(0{★}^{L}0\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}=\frac{h\left(Laf\right)-h\left(0\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}=\frac{ah\left(Lf\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}$
$=a\frac{h\left(Lf\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}=a\frac{h\left(Lf\right)-h\left(0\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}=a\frac{h\left(f{★}^{L}0\right)-h\left(0{★}^{L}0\right)}{h\left(1{★}^{L}0\right)-h\left(0{★}^{L}0\right)}=a\left(h{|}^{0}L\right)\left(f\right)$
Cohomogenity using cohomogenity for h
$\left(h{|}^{1}L\right)(1+af)=\frac{h\left(\right(1+af\left){★}^{L}1\right)-h\left(0{★}^{L}1\right)}{h\left(1{★}^{L}1\right)-h\left(0{★}^{L}1\right)}=\frac{h(L+aLf+1-L)-h(1-L)}{h\left(1\right)-h(1-L)}$
$=\frac{h\left(1_{a}Lf\right)-h(1-L)}{1-h(1-L)}=\frac{(1-a+ah(1+Lf\left)\right)-h(1-L)}{1-h(1-L)}$
$=\frac{1-a+ah(1+Lf)-h(1-L)+ah(1-L)-ah(1-L)}{1-h(1-L)}$
$=\frac{1-h(1-L)}{1-h(1-L)}-\frac{a-ah(1-L)}{1-h(1-L)}+\frac{ah(1+Lf)-ah(1-L)}{1-h(1-L)}$
$=\frac{1-h(1-L)}{1-h(1-L)}-a\frac{1-h(1-L)}{1-h(1-L)}+a\frac{h(1+Lf)-h(1-L)}{1-h(1-L)}$
$=1-a+a\frac{h(1+Lf)-h(1-L)}{h\left(1\right)-h(1-L)}=1-a+a\frac{h(L+Lf+(1-L\left)\right)-h(1-L)}{h(L+(1-L\left)\right)-h(1-L)}$
$=1-a+a\frac{h\left(\right(1+f\left){★}^{L}1\right)-h\left(0{★}^{L}1\right)}{h\left(1{★}^{L}1\right)-h\left(0{★}^{L}1\right)}=1-a+a\left(h{|}^{1}L\right)(1+f)$

Now for mixtures, we’ll verify homogenity, 1-Lipschitzness, cohomogenity, C-additivity, and crispness.

Homogenity:
$\left(E_{\zeta }{h}_i\right)\left(af\right)=E_{\zeta }\left(h_i\right(af\left)\right)=E_{\zeta }\left(a{h}_i\right(f\left)\right)=a{E}_{\zeta }\left(h_i\right(f\left)\right)=a\left(E_{\zeta }{h}_i\right)\left(f\right)$
1-Lipschitz:
$|\left(E_{\zeta }{h}_i\right)\left(f\right)-\left(E_{\zeta }{h}_i\right)\left(f^{\prime }\right)|=|E_{\zeta }\left(h_i\right(f\left)\right)-E_{\zeta }\left(h_i\right(f^{\prime }\left)\right)|$
$\le E_{\zeta }|h_i\left(f\right)-h_i\left(f^{\prime }\right)|\le E_{\zeta }d(f,f^{\prime })=d(f,f^{\prime })$
Cohomogenity:
$\left(E_{\zeta }{h}_i\right)(1+af)=E_{\zeta }\left(h_i\right(1+af\left)\right)=E_{\zeta }(1-a+a{h}_i(1+f\left)\right)$
$=1-a+a{E}_{\zeta }\left(h_i\right(1+f\left)\right)=1-a+a\left(E_{\zeta }{h}_i\right)(1+f)$
C-additivity:
$\left(E_{\zeta }{h}_i\right)\left(c\right)=E_{\zeta }\left(h_i\right(c\left)\right)=E_{\zeta }\left(c\right)=c$
Crispness: Observe that both homogenity and C-additivity are preserved, and crispness is equivalent to the conjunction of the two.

Now for deterministic pushforwards, we’ll verify homogenity, 1-Lipschitzness, cohomogenity, C-additivity, crispness, and sharpness.

Homogenity:
$\left(g_{\ast }\right(h\left)\right)\left(af\right)=h\left(\right(af)\circ g)=h\left(a\right(f\circ g\left)\right)=ah(f\circ g)=a\left(g_{\ast }\right(h\left)\right)\left(f\right)$
1-Lipschitzness:
$|\left(g_{\ast }\right(h\left)\right)\left(f\right)-\left(g_{\ast }\right(h\left)\right)\left(f^{\prime }\right)|=|h(f\circ g)-h(f^{\prime }\circ g)|\le d(f\circ g,f^{\prime }\circ g)\le d(f,f^{\prime })$
Cohomogenity:
$\left(g_{\ast }\right(h\left)\right)(1+af)=h\left(\right(1+af)\circ g)=h(1+a(f\circ g\left)\right)=1-a+ah(1+(f\circ g\left)\right)$
$=1-a+ah\left(\right(1+f)\circ g)=1-a+a\left(g_{\ast }\right(h\left)\right)(1+f)$
C-additivity:
$\left(g_{\ast }\right(h\left)\right)\left(c\right)=h(c\circ g)=h\left(c\right)=c$
Crispness: Both homogenity and C-additivity are preserved, so crispness is preserved too.

Sharpness:
$\left(g_{\ast }\right(h\left)\right)\left(f\right)=h(f\circ g)={inf}_{x\in C}f\left(g\right(x\left)\right)={inf}_{y\in g\left(C\right)}f\left(y\right)$
And $g\left(C\right)$ is the image of a compact set, so it’s compact. And we’re done!

Proposition 11: The inf of two infradistributions is always an infradistribution, and inf preserves the infradistribution properties indicated by the diagram at the start of this section.

We’ll first verify the infradistribution properties of the inf, and then show it preserves the indicated properties if both components have them.

We must check monotonicity, concavity, normalization, Lipschitzness, and compact almost-support. For monotonicity, if $f^{\prime }\ge f$, then
$inf(h_1,h_2)\left(f^{\prime }\right)=inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)\ge inf\left(h_1\right(f),h_2(f\left)\right)=inf(h_1,h_2)\left(f\right)$
This was done by monotonicity for the components. For concavity,
$inf(h_1,h_2)(pf+(1-p\left)f^{\prime }\right)=inf\left(h_1\right(pf+(1-p)f^{\prime }),h_2(pf+(1-p)f^{\prime }\left)\right)$
$\ge inf\left(p{h}_1\right(f)+(1-p\left)h_1\right(f^{\prime }),p{h}_2(f)+(1-p\left)h_2\right(f^{\prime }\left)\right)$
$\ge inf\left(p{h}_1\right(f),p{h}_2(f\left)\right)+inf\left(\right(1-p\left)h_1\right(f^{\prime }),(1-p\left)h_2\right(f^{\prime }\left)\right)$
$=pinf\left(h_1\right(f),h_2(f\left)\right)+(1-p)inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)$
$=pinf(h_1,h_2)\left(f\right)+(1-p)inf(h_1,h_2)\left(f^{\prime }\right)$
The first $\ge$ happened because $h_1$ and $h_2$ are concave, the second is because $inf(a+b,c+d)\ge inf(a,c)+inf(b,d)$.

For normalization,
$inf(h_1,h_2)\left(1\right)=inf\left(h_1\right(1),h_2(1\left)\right)=inf(1,1)=1$
And the same argument applies to 0, so the inf is normalized.

For Lipschitzness, the inf of two Lipschitz functions is Lipschitz.

That just leaves compact almost-support. Fix an arbitary $ϵ$, and get a $C_{ϵ}^1$ compact $ϵ$-almost-support for $h_1$, and a $C_{ϵ}^2$ for $h_2$. We will show that $C_{ϵ}^1\cup C_{ϵ}^2$ is a compact $ϵ$-almost-support for $inf(h_1,h_2)$. It’s compact because it’s a finite union of compact sets.

Now, let $f$ and $f^{\prime }$ agree on $C_{ϵ}^1\cup C_{ϵ}^2$. We can go:
$|inf(h_1,h_2)\left(f\right)-inf(h_1,h_2)\left(f^{\prime }\right)|=|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|$
There are four possible cases for evaluating this quantity. In case 1, $h_1\left(f\right)\le h_2\left(f\right)$ and $h_1\left(f^{\prime }\right)\le h_2\left(f^{\prime }\right)$. Then our above term turns into $|h_1\left(f\right)-h_1\left(f^{\prime }\right)|$. However, since $f$ and $f^{\prime }$ agree on $C_{ϵ}^1\cup C_{ϵ}^2$, they must agree on $C_{ϵ}^1$, and only have expectations $\le ϵd(f,f^{\prime })$ apart. Case 2 where $h_1\left(f\right)\ge h_2\left(f\right)$ and $h_1\left(f^{\prime }\right)\ge h_2\left(f^{\prime }\right)$ is symmetric and can be disposed of by a nearly identical argument, we just do it with $h_2$ and $C_{ϵ}^2$.

Case 3 where $h_1\left(f\right)<h_2\left(f\right)$ and $h_1\left(f^{\prime }\right)>h_2\left(f^{\prime }\right)$ takes a slightly fancier argument. We can go:
$-ϵd(f,f^{\prime })<h_1\left(f\right)-h_1\left(f^{\prime }\right)<h_1\left(f\right)-h_2\left(f^{\prime }\right)<h_2\left(f\right)-h_2\left(f^{\prime }\right)<ϵd(f,f^{\prime })$
The end inequalities are because $f$ and $f^{\prime }$ agree on the $ϵ$-almost-supports of $h_1$ and $h_2$, respectively, from agreeing on the union. The two inner inequalities are derived from the assumed inequalities in Case 3.
Thus,
$|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|=|h_1\left(f\right)-h_2\left(f^{\prime }\right)|<ϵd(f,f^{\prime })$
Case 4 where the assumed starting inequalities go in the other direction is symmetric. So, no matter which infradistributions are lower in the two infs, we have
$|inf(h_1,h_2)\left(f\right)-inf(h_1,h_2)\left(f^{\prime }\right)|=|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|<ϵd(f,f^{\prime })$
And we’re done, we made a compact almost-support for $inf(h_1,h_2)$ assuming an arbitrary $ϵ$. So the inf of two infradistributions is a infradistribution.

Now to verify homogenity, 1-Lipschitzness, cohomogenity, C-additivity, crispness, and sharpness preservation.

Homogenity:
$inf(h_1,h_2)\left(af\right)=inf\left(h_1\right(af),h_2(af\left)\right)=inf\left(a{h}_1\right(f),a{h}_2(f\left)\right)$
$=ainf\left(h_1\right(f),h_2(f\left)\right)=ainf(h_1,h_2)\left(f\right)$
1-Lipschitzness:
$|inf(h_1,h_2)\left(f\right)-inf(h_1,h_2)\left(f^{\prime }\right)|=|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|$
Now we can split into four cases. In cases 1 and 2 where the infs turn into $h_1\left(f\right),h_1\left(f^{\prime }\right)$ (and same for $h_2$ in case 2), we have:
$|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|=|h_1\left(f\right)-h_1\left(f^{\prime }\right)|\le d(f,f^{\prime })$
(and same for $h_2$), and we’re done with those cases. In cases 3 and 4 where the infs turn into $h_1\left(f\right),h_2\left(f^{\prime }\right)$ (and vice-versa for case 4), we have:
$-d(f,f^{\prime })\le h_1\left(f\right)-h_1\left(f^{\prime }\right)<h_1\left(f\right)-h_2\left(f^{\prime }\right)<h_2\left(f\right)-h_2\left(f^{\prime }\right)\le d(f,f^{\prime })$
Because $h_1$ and $h_2$ are 1-Lipschitz. Thus,
$|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|=|h_1\left(f\right)-h_2\left(f^{\prime }\right)|\le d(f,f^{\prime })$
A symmetric argument works for case 4. So, no matter what,
$|inf\left(h_1\right(f),h_2(f\left)\right)-inf\left(h_1\right(f^{\prime }),h_2(f^{\prime }\left)\right)|\le d(f,f^{\prime })$
And we’re done, the inf is 1-Lipschitz too.

Cohomogenity:
$inf(h_1,h_2)(1+af)=inf\left(h_1\right(1+af),h_2(1+af\left)\right)$
$=inf(1-a+a{h}_1(1+f),1-a+a{h}_2(1+f\left)\right)$
$=1-a+ainf\left(h_1\right(1+f),h_2(1+f\left)\right)$
$=1-a+ainf(h_1,h_2)(1+f)$
C-additivity:
$inf(h_1,h_2)\left(c\right)=inf\left(h_1\right(c),h_2(c\left)\right)=inf(c,c)=c$
Crispness: Homogenity and C-additivity are both preserved, so crispness is preserved.

Sharpness:
$inf(h_1,h_2)\left(f\right)=inf\left(h_1\right(f),h_2(f\left)\right)=inf\left({inf}_{x\in C_1}f\right(x),{inf}_{x\in C_2}f(x\left)\right)={inf}_{x\in C_1\cup C_2}f\left(x\right)$
And we’re done.

Proposition 12: $E_{inf(H_1,H_2)}\left(f\right)=inf\left(E_{H_1}\right(f),E_{H_2}(f\left)\right)$

$E_{inf(H_1,H_2)}\left(f\right)={inf}_{(m,b)\in inf(H_1,H_2)}m\left(f\right)+b={inf}_{(m,b)\in H_1\cup H_2}m\left(f\right)+b$
$=inf\left({inf}_{(m,b)\in H_1}\right(m\left(f\right)+b),{inf}_{(m,b)\in H_2}(m\left(f\right)+b\left)\right)$
$=inf\left(E_{H_1}\right(f),E_{H_2}(f\left)\right)$

Proposition 13: If a family of infradistributions $\{h_i{\}}_{i\in I}$ has a shared upper bound on the Lipschitz constant, and for all $ϵ$, there is a compact set $C_{ϵ}$ that is an $ϵ$-almost support for all $h_i$, then ${inf}_i{h}_i$, defined as $\left({inf}_i{h}_i\right)\left(f\right):={inf}_i\left(h_i\right(f\left)\right)$, is an infradistribution. Further, for all conditions listed in the table, if all the $h_i$ fulfill them, then ${inf}_i{h}_i$ fulfills the same property.

We’ll first verify the infradistribution properties of the infinite inf, and then show it preserves the indicated properties if all components have them.

We must check monotonicity, concavity, normalization, Lipschitzness, and compact almost-support. For monotonicity, if $f^{\prime }\ge f$, then
$\left({inf}_i{h}_i\right)\left(f^{\prime }\right)={inf}_i\left(h_i\right(f^{\prime }\left)\right)\ge {inf}_i\left(h_i\right(f\left)\right)=\left({inf}_i{h}_i\right)\left(f\right)$
This was done by monotonicity for all components. For concavity,
$\left({inf}_i{h}_i\right)(pf+(1-p\left)f^{\prime }\right)={inf}_i\left(h_i\right(pf+(1-p)f^{\prime }\left)\right)\ge {inf}_i\left(p{h}_i\right(f)+(1-p\left)h_i\right(f^{\prime }\left)\right)$
$\ge {inf}_i\left(p{h}_i\right(f\left)\right)+{inf}_i\left(\right(1-p\left)h_i\right(f^{\prime }\left)\right)=p{inf}_i\left(h_i\right(f\left)\right)+(1-p){inf}_i\left(h_i\right(f^{\prime }\left)\right)$
$=p\left({inf}_i{h}_i\right)\left(f\right)+(1-p)\left({inf}_i{h}_i\right)\left(f^{\prime }\right)$
The first $\ge$ happened because $h_1$ and $h_2$ are concave, the second is because ${inf}_i(a_i+b_i)\ge {inf}_i\left(a_i\right)+{inf}_i\left(b_i\right)$.

For normalization,
$\left({inf}_i{h}_i\right)\left(1\right)={inf}_i\left(h_i\right(1\left)\right)={inf}_i\left(1\right)=1$
And the same argument applies to 0, so the inf is normalized.

For Lipschitzness, let ${\lambda }^{\odot }$ be your uniform upper bound on the Lipschitz constants of the $h_i$. Then,
$|\left({inf}_i{h}_i\right)\left(f\right)-\left({inf}_i{h}_i\right)\left(f^{\prime }\right)|=|{inf}_i\left(h_i\right(f\left)\right)-{inf}_i\left(h_i\right(f^{\prime }\left)\right)|$
And then, for all the $h_i$, they only think those functions differ by ${\lambda }^{\odot }d(f,f^{\prime })$ or less, and the same property applies to the inf by picking a $h_i$ and $h_{i^{\prime }}$ that very very nearly attain the two minimums, and showing that if the infinimums were $>{\lambda }^{\odot }d(f,f^{\prime })$ apart, you could have $h_i\left(f^{\prime }\right)$ appreciably undershoot $h_{i^{\prime }}\left(f^{\prime }\right)$, and in fact, undershoot ${inf}_i\left(h_i\right(f^{\prime }\left)\right)$, which is impossible. Thus,
$|{inf}_i\left(h_i\right(f\left)\right)-{inf}_i\left(h_i\right(f^{\prime }\left)\right)|\le {\lambda }^{\odot }d(f,f^{\prime })$
And we’re done.

That just leaves compact almost-support. Fix an arbitary $ϵ$. We know there is some $C_{ϵ}$ that is a compact $ϵ$-almost-support for all the $h_i$. We will show that $C_{ϵ}$ is an $ϵ$-almost-support for ${inf}_i{h}_i$.

Let $f$ and $f^{\prime }$ agree on $C_{ϵ}$. We can go:
$|\left({inf}_i{h}_i\right)\left(f\right)-\left({inf}_i{h}_i\right)\left(f^{\prime }\right)|=|{inf}_i\left(h_i\right(f\left)\right)-{inf}_i\left(h_i\right(f^{\prime }\left)\right)|$
Pick a $h_i$ and $h_{i^{\prime }}$ that very very very nearly attain the inf. Then we can approximately reexpress this quantity as:
$|inf\left(h_i\right(f),h_{i^{\prime }}(f\left)\right)-inf\left(h_i\right(f^{\prime }),h_{i^{\prime }}(f^{\prime }\left)\right)|$
We’re approximately in a case where $h_i\left(f\right)\le h_{i^{\prime }}\left(f\right)$ and $h_i\left(f^{\prime }\right)\ge h_{i^{\prime }}\left(f^{\prime }\right)$, so we can go:
$-ϵd(f,f^{\prime })\le h_i\left(f\right)-h_i\left(f^{\prime }\right)<h_i\left(f\right)-h_{i^{\prime }}\left(f^{\prime }\right)<h_{i^{\prime }}\left(f\right)-h_{i^{\prime }}\left(f^{\prime }\right)\le ϵd(f,f^{\prime })$
The end inequalities are because $f$ and $f^{\prime }$ agree on the $ϵ$-almost-support of $h_i$ and $h_{i^{\prime }}$. The two inner inequalities are derived from the assumed inequalities in our case. Thus,
$|{inf}_i\left(h_i\right(f\left)\right)-{inf}_i\left(h_i\right(f^{\prime }\left)\right)|\simeq |h_i\left(f\right)-h_{i^{\prime }}\left(f^{\prime }\right)|\le ϵd(f,f^{\prime })$
And we’re done, we made a compact almost-support for ${inf}_i{h}_i$ assuming an arbitrary $ϵ$. So the inf of this family of infradistributions is a infradistribution.

Now to verify homogenity, 1-Lipschitzness, cohomogenity, C-additivity, crispness, and sharpness preservation.

Homogenity:
$\left({inf}_i{h}_i\right)\left(af\right)={inf}_i\left(h_i\right(af\left)\right)={inf}_i\left(a{h}_i\right(f\left)\right)$
$=a{inf}_i\left(h_i\right(f\left)\right)=a\left({inf}_i{h}_i\right)\left(f\right)$
1-Lipschitzness: Same as the Lipschitz argument, everyone has a Lipschitz constant of 1, so the inf has the same Lipschitz constant.

Cohomogenity:
$\left({inf}_i{h}_i\right)(1+af)={inf}_i\left(h_i\right(1+af\left)\right)$
$={inf}_i(1-a+a{h}_i(1+f\left)\right)$
$=1-a+a{inf}_i\left(h_i\right(1+f\left)\right)$
$=1-a+a\left({inf}_i{h}_i\right)(1+f)$
C-additivity:
$\left({inf}_i{h}_i\right)\left(c\right){inf}_i\left(h_i\right(c\left)\right)={inf}_i\left(c\right)=c$
Crispness: Homogenity and C-additivity are both preserved, so crispness is preserved.

Sharpness:
$\left({inf}_i{h}_i\right)\left(f\right)=inf\left(h_i\right(f\left)\right)={inf}_i\left({inf}_{x\in C_i}f\right(x\left)\right)={inf}_{x\in {\bigcup }_i{C}_i}f\left(x\right)={inf}_{x\in \overline{{\bigcup }_i{C}_i}}f\left(x\right)$
We do have to check whether or not ${\bigcup }_i{C}_i$ is compact, however. We’ll start by showing that for an arbitrary $C_i$, any compact set $K$ where $C_i⊈K$ can’t be an $ϵ$-support of $h_i$ for any $ϵ<1$. The proof proceeds as follows:

Let $x^{\ast }$ be some point in $C_i$ but not in $K$. It must be some finite distance away from $K$. Craft a continuous function $f^0$ supported on $K\cup \left\{x^{\ast }\right\}$. $f^0$ is 1 on $K$ and 0 on $\left\{x^{\ast }\right\}$. Use the Tietze extension theorem to extend $f^0$ to all of $X$. Then
$|h_i\left(f^0\right)-h_i\left(1\right)|=|{inf}_{x\in C_i}{f}^0\left(x\right)-{inf}_{x\in C_i}1|=|f^0\left(x^{\ast }\right)-1|=|0-1|=1$
However, $f^0$ and 1 agree on $K$, so $K$ can’t be an $ϵ$-almost-support for any $ϵ<1$.

Thus, in order for there to be a compact set $C_{ϵ}$ that’s an $ϵ$-almost-support for all $h_i$, it must be that $\forall i:C_i\subseteq C_{ϵ}$. Then
$\overline{{\bigcup }_i{C}_i}\subseteq C_{ϵ}$
because all the $C_i$ are in it and $C_{ϵ}$ is closed. So, the closure of our union is a closed subset of a compact set and thus is compact, so ${inf}_i{h}_i$ is minimizing over a compact set and thus is crisp.

Proposition 14: If $sup(h_1,h_2)\left(0\right)=0$ and $sup(h_1,h_2)\left(1\right)=1$, then the supremum is an infradistribution.

The supremum is defined as:
$sup(h_1,h_2)\left(f\right)={sup}_{f_1,f_2,p:p{f}_1+(1-p)f_2\le f}p{h}_1\left(f_2\right)+(1-p)h_2\left(f_2\right)$
We’ll verify the infradistribution properties of the sup.

We must check monotonicity, concavity, normalization, Lipschitzness, and compact almost-support. For monotonicity, if $f^{\prime }\ge f$, then
$sup(h_1,h_2)\left(f\right)={sup}_{f_1,f_2,p:p{f}_1+(1-p)f_2\le f}p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$
$\le {sup}_{f_1,f_2,p:p{f}_1+(1-p)f_2\le f^{\prime }}p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)=sup(h_1,h_2)\left(f^{\prime }\right)$
This was done by $f^{\prime }\ge f$ so there’s more options available. For concavity,
$qsup(h_1,h_2)\left(f\right)+(1-q)sup(h_1,h_2)\left(f^{\prime }\right)$
$=q{sup}_{f_1,f_2,p:p{f}_1+(1-p)f_2\le f}p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$
$+(1-q){sup}_{f_1^{\prime },f_2^{\prime },p^{\prime }:p^{\prime }{f}_1^{\prime }+(1-p^{\prime })f_2^{\prime }\le f}{p}^{\prime }{h}_1\left(f_1^{\prime }\right)+(1-p^{\prime })h_2\left(f_2^{\prime }\right)$
Pick your $f_1,f_2,f_1^{\prime },f_2^{\prime },p,p^{\prime }$ that very very very nearly attain the supremum.
$\simeq qp{h}_1\left(f_1\right)+q(1-p)h_2\left(f_2\right)+(1-q)p^{\prime }{h}_1\left(f_1^{\prime }\right)+(1-q)(1-p^{\prime })h_2\left(f_2^{\prime }\right)$
$=(qp+(1-q\left)p^{\prime }\right)\left(\frac{qp}{qp+(1-q)p^{\prime }}{h}_1\right(f_1)+\frac{(1-q)p^{\prime }}{qp+(1-q)p^{\prime }}{h}_1(f_1^{\prime }\left)\right)$
$+\left(q\right(1-p)+(1-q\left)\right(1-p^{\prime }\left)\right)\left(\frac{q(1-p)}{q(1-p)+(1-q)(1-p^{\prime })}{h}_2\right(f_2)+\frac{(1-q)(1-p^{\prime })}{q(1-p)+(1-q)(1-p^{\prime })}{h}_2(f_2^{\prime }\left)\right)$
$\le (qp+(1-q\left)p^{\prime }\right)h_1(\frac{qp}{qp+(1-q)p^{\prime }}{f}_1+\frac{(1-q)p^{\prime }}{qp+(1-q)p^{\prime }}{f}_1^{\prime })$
$+\left(q\right(1-p)+(1-q\left)\right(1-p^{\prime }\left)\right)h_2(\frac{q(1-p)}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2+\frac{(1-q)(1-p^{\prime })}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2^{\prime })$
Also, we can verify that:
$(qp+(1-q\left)p^{\prime }\right)(\frac{qp}{qp+(1-q)p^{\prime }}{f}_1+\frac{(1-q)p^{\prime }}{qp+(1-q)p^{\prime }}{f}_1^{\prime })$
$+\left(q\right(1-p)+(1-q\left)\right(1-p^{\prime }\left)\right)(\frac{q(1-p)}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2+\frac{(1-q)(1-p^{\prime })}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2^{\prime })$
$=qp{f}_1+(1-q)p^{\prime }{f}_1^{\prime }+q(1-p)f_2+(1-q)(1-p^{\prime })f_2^{\prime }$
$=q(p{f}_1+(1-p\left)f_2\right)+(1-q)(p^{\prime }{f}_1^{\prime }+(1-p^{\prime }\left)f_2^{\prime }\right)\le qf+(1-q)f^{\prime }$
Therefore, it is a suitable parameter and pair of functions to lower bound
$qf+(1-q)f^{\prime }$. Accordingly
$(qp+(1-q\left)p^{\prime }\right)h_1(\frac{qp}{qp+(1-q)p^{\prime }}{f}_1+\frac{(1-q)p^{\prime }}{qp+(1-q)p^{\prime }}{f}_1^{\prime })$
$+\left(q\right(1-p)+(1-q\left)\right(1-p^{\prime }\left)\right)h_2(\frac{q(1-p)}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2+\frac{(1-q)(1-p^{\prime })}{q(1-p)+(1-q)(1-p^{\prime })}{f}_2^{\prime })$
$\le {sup}_{p^{\ast },f_1^{\ast },f_2^{\ast }:p^{\ast }{f}_1^{\ast }+(1-p^{\ast })f_2^{\ast }\le qf+(1-q)f^{\prime }}{p}^{\ast }{h}_1\left(f_1^{\ast }\right)+(1-p^{\ast })h_2\left(f_2^{\ast }\right)=sup(h_1,h_2)(qf+(1-q\left)f^{\prime }\right)$
Putting all this together, and picking better and better approximations to the two suprema, we can conclude that:
$qsup(h_1,h_2)\left(f\right)+(1-q)sup(h_1,h_2)\left(f^{\prime }\right)\le sup(h_1,h_2)(qf+(1-q\left)f^{\prime }\right)$
And we have concavity.

For normalization, we’re assuming it holds at the start.

Lipschitzness takes a slightly more involved argument. Pick two functions $f$ and $f^{\prime }$, and without loss of generality, assume $sup(h_1,h_2)\left(f\right)\ge sup(h_1,h_2)\left(f^{\prime }\right)$. Now, what we can do is pick a $p$, $f_1$ and $f_2$ which approximately obtain the defining supremum for $f$, so we have:
$sup(h_1,h{)}_2)\left(f\right)\simeq p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$
Now, we can note two things. First,
$p(f_1-d(f,f^{\prime }\left)\right)+(1-p)(f_2-d(f,f^{\prime }\left)\right)=(p{f}_1+(1-p\left)f_2\right)-d(f,f^{\prime })\le f-d(f,f^{\prime })\le f^{\prime }$
Therefore, the same $p$, and $f_1-d(f,f^{\prime })$, and $f_2-d(f,f^{\prime })$ are suitable things to lower-bound the value of $sup(h_1,h_2)\left(f^{\prime }\right)$. In particular, we have:
$sup(h_1,h_2)\left(f^{\prime }\right)={sup}_{q,f_1^{\prime },f_2^{\prime }:q{f}_1^{\prime }+(1-q)f_2^{\prime }\le f^{\prime }}q{h}_1\left(f_1^{\prime }\right)+(1-q)h_2\left(f_2^{\prime }\right)$
$\ge p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)$
Also, we have the result that:
$p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)-p{h}_1(f_1-d(f,f^{\prime }\left)\right)-(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)$
$=|\left(p{h}_1\right(f_1)+(1-p\left)h_2\right(f_2\left)\right)-\left(p{h}_1\right(f_1-d(f,f^{\prime }))-(1-p\left)h_2\right(f_2-d(f,f^{\prime })\left)\right)|$
$\le |p{h}_1\left(f_1\right)-p{h}_1(f_1-d(f,f^{\prime }\left)\right)|+|(1-p)h_2\left(f_2\right)-(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)|$
$=p|h_1\left(f_1\right)-h_1(f_1-d(f,f^{\prime }\left)\right)|+(1-p)|h_2\left(f_2\right)-h_2(f_2-d(f,f^{\prime }\left)\right)|$
$\le p({\lambda }_1^{\odot }\cdot d(f,f^{\prime }\left)\right)+(1-p)({\lambda }_2^{\odot }\cdot d(f,f^{\prime }\left)\right)\le max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
Because of Lipschitzness of $h_1$ and $h_2$. Now we can begin showing our inequalities. So, we’ve shown that:
$sup(h_1,h_2)\left(f^{\prime }\right)\ge p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)$
Therefore,
$p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)-sup(h_1,h_2)\left(f^{\prime }\right)\le 0$
With this result, we can go:
$max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
$\ge max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })+p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
Let’s save this result for a bit later.
Also, we had:
$p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)-p{h}_1(f_1-d(f,f^{\prime }\left)\right)-(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)\le max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
And we also picked $p$ and $f_1$ and $f_2$ to approximately attain the supremum, so we know:
$p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)\simeq sup(h_1,h_2)\left(f\right)$
Therefore, we approximately have:
$sup(h_1,h_2)\left(f\right)-p{h}_1(f_1-d(f,f^{\prime }\left)\right)-(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)\le max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
Reshuffling this around a bit, we have:
$max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })+p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)\ge sup(h_1,h_2)\left(f\right)$
Using this with our saved result, we can get:
$max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
$\ge max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })+p{h}_1(f_1-d(f,f^{\prime }\left)\right)+(1-p)h_2(f_2-d(f,f^{\prime }\left)\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
$\ge sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)\ge 0$
That last inequality was because we assumed at the start without loss of generality that $f$ got an equal or higher expectation than $f^{\prime }$.
Therefore, we have our result that, in general,
$|sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)|\le max({\lambda }_1^{\odot },{\lambda }_2^{\odot })d(f,f^{\prime })$
And thus, the supremum of two Lipschitz infradistributions is Lipschitz. That just leaves compact almost-support, which is quite tricky to show.

Fix an arbitary $ϵ$, and get a $C_{ϵ}^1$ compact $ϵ$-almost-support for $h_1$, and a $C_{ϵ}^2$ for $h_2$. We will show that $C_{ϵ}^1\cup C_{ϵ}^2$ is a compact $ϵ$-almost-support for $sup(h_1,h_2)$. It’s compact because it’s a finite union of compact sets.

Now, let $f$ and $f^{\prime }$ agree on $C_{ϵ}^1\cup C_{ϵ}^2$. Without loss of generality, assume that $sup(h_1,h_2)\left(f\right)\ge sup(h_1,h_2)\left(f^{\prime }\right)$ (if not, flip $f$ and $f^{\prime }$). We’ll show that they have similar expectations by showing that $sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)$ is below a small number (we already know that it’s above 0 by our without-loss-of-generality assumption).

We can go:
$sup(h_1,h_2)\left(f\right)={sup}_{p,f_1,f_2:p{f}_1+(1-p)f_2\le f}p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)\simeq p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$
Where we picked a particular $p,f_1,f_2$ spectacularly close to the highest possible value s.t. $p{f}_1+(1-p)f_2\le f$. In particular, if $p$ is 0 or 1, we can ensure that $f_1$ or $f_2$ is $f$ itself, by monotonicity of $h_1$ or $h_2$ respectively.

For successive arguments, we need $p\in (0,1)$ so we have to address those endpoints. Assume $p=1$. Then, $sup(h_1,h_2)\left(f\right)\simeq h_1\left(f\right)$. Then, we have:
$sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
$\simeq h_1\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
$\le h_1\left(f^{\prime }\right)+ϵd(f,f^{\prime })-sup(h_1,h_2)\left(f^{\prime }\right)\le ϵd(f,f^{\prime })$
The way this works is our substitution, and then using that $f$ and $f^{\prime }$ are identical on $C_{ϵ}^1\cup C_{ϵ}^2$, and so are identical on $C_{ϵ}^1$, which is $ϵ$-almost-support of $h_1$, we can upper-bound $h_1\left(f\right)$ with $h_1\left(f^{\prime }\right)+ϵd(f,f^{\prime })$. And then, we just use that $h_1\left(f^{\prime }\right)\le sup(h_1,h_2)\left(f^{\prime }\right)$. If $p=0$, the exact same argument works, just with $h_2$ and $C_{ϵ}^2$ instead. That leaves the case where $p\in (0,1)$, which requires far more involved arguments.

As a recap, we’re assuming that $sup(h_1,h_2)\left(f\right)\ge sup(h_1,h_2)\left(f^{\prime }\right)$, and that $sup(h_1,h_2)\left(f\right)\simeq p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$, and $p\in (0,1)$. Now, we’re going to pick out a continuous function with some special properties, so let the set-valued function $\psi :X\to R^2$ be defined as: If $x\in C_{ϵ}^1\cup C_{ϵ}^2$, then $\psi \left(x\right)=\left(f_1\right(x),f_2(x\left)\right)$. Otherwise, $\psi \left(x\right)$ equals the intersection of:
$\left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
and
$\left\{\right(y,z)|py+(1-p)z\le f^{\prime }(x\left)\right\}$
We’ll find a continuous selection of this set-valued function, so let’s start checking the properties needed to invoke the Michael selection theorem. We need that $X$ is paracompact (all polish spaces are paracompact, check), that $R^2$ is a Banach space (check), that for all $x$, $\psi \left(x\right)$ is convex (it’s either a single point or the intersection of a rectangle and a half-space, which is convex in both cases), closed (yup, it’s either a point or the intersection of two closed sets, ie closed), nonempty, and lower-hemicontinuous.

Nonemptiness isn’t too bad to show. It’s nonempty for all points in our compact set of interest (the set consisting of a single point), and for x not in said set, $\left(f_1\right(x)-d(f,f^{\prime }),f_2(x)-d(f,f^{\prime }\left)\right)$ witnesses the nonemptiness, because:
$p\left(f_1\right(x)-d(f,f^{\prime }\left)\right)+(1-p)\left(f_2\right(x)-d(f,f^{\prime }\left)\right)=p{f}_1\left(x\right)+(1-p)f_2\left(x\right)-d(f,f^{\prime })$
$\le f\left(x\right)-d(f,f^{\prime })\le f^{\prime }\left(x\right)$
Lower-hemicontinuity is much more challenging to establish. Again, we have a sequence $x_n$ limiting to $x$, a point $(y,z)\in \psi \left(x\right)$, and we must find a subsequence $(y_m,z_m)\in \psi \left(x_m\right)$ which limits to $(y,z)$.

We can divide into three cases. In the first case, $x$ lies in $C_{ϵ}^1\cup C_{ϵ}^2$, and infinitely many members of the sequence lie in said set. In particular, since $x$ lies in the compact set, the $(y,z)$ pair associated with it must be $\left(f_1\right(x),f_2(x\left)\right)$. Then we can isolate that particular subsequence that lies in the compact set, and have $(y_m,z_m)$ be $\left(f_1\right(x_m),f_2(x_m\left)\right)$, which, by continuity of $f_1$ and $f_2$, and the definition of $\psi \left(x_m\right)$ for $x_m$ in the compact set, lie in $\psi \left(x_m\right)$ and limit to $(y,z)$ ie $\left(f_1\right(x),f_2(x\left)\right)$.

In preparation for the second and third cases, we’ll show that the function ${\psi }^{\prime }:X\to R^2$ which just takes the second branch of the $\psi$ function is continuous w.r.t. the Hausdorff-metric. Ie, for all $x$,
${\psi }^{\prime }\left(x\right):=\left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
$\cap \left\{\right(y,z)|py+(1-p)z\le f^{\prime }(x\left)\right\}$
is continuous when the space of compact subsets of $R^2$ is equipped with a Hausdorff distance.

Accordingly, let $x_m$ limit to $x$. Our task is to show that, no matter how tiny of a number you name, you can find a tail of the $x_m$ sequence where the Hausdorff distance between ${\psi }^{\prime }\left(x_m\right)$ and ${\psi }^{\prime }\left(x\right)$ is that tiny.

Specifically, we’ll show that for all $\delta$, there is some $m$ where all later $x_m$ have ${\psi }^{\prime }\left(x_m\right)$ within $\frac{2\delta }{p}+\frac{2\delta }{1-p}+4\delta$ Hasdorff distance of ${\psi }^{\prime }\left(x\right)$. Because $p\in (0,1)$ and we can shrink $\delta$ to 0, this shows that the function ${\psi }^{\prime }$ is continuous in Hausdorff-distance.

Because $f_1$ and $f_2$ and $f^{\prime }$ are continuous functions, there’s some very very large $m$ where $f_1$, $f_2$, and $f^{\prime }$ will only vary by $\delta$ from that point forward, regardless of which $\delta$ you pick. Pick some arbitrary $(y_m,z_m)\in {\psi }^{\prime }\left(x_m\right)$. We’ll show that it’s close to a $(y,z)\in {\psi }^{\prime }\left(x\right)$, and the argument will only depend on distances, not position in sequence, so we can flip it to show the other half of Hausdorff-distance (all points in ${\psi }^{\prime }\left(x\right)$ are close to a point in ${\psi }^{\prime }\left(x_m\right)$).

We can divide into four possible cases. In cases 1 and 2, we have the following property holding.
$y_m\ge f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta$
With the negation for cases 3 and 4.

And in cases 1 and 3, we have:
$z_m\ge f_2\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{1-p}+\delta$
With the negation for cases 2 and 4.

In cases 1 and 2, you can let your selected $y$ point be $y_m-\frac{2\delta }{p}$. We have the result that $y\in \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]$, because:
$f_1\left(x\right)+d(f,f^{\prime })\ge f_1\left(x_m\right)-\delta +d(f,f^{\prime })\ge y_m-\delta \ge y_m-\frac{2\delta }{p}=y$
$\ge f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta -\frac{2\delta }{p}=f_1\left(x_m\right)-d(f,f^{\prime })+\delta \ge f_1\left(x\right)-d(f,f^{\prime })$
In order, the first inequality is because $f_1$ only varies by $\delta$ over such tiny distances due to continuity of $f_1$, the second inequality is $y_m$ being paired with something to be in ${\psi }^{\prime }\left(x_m\right)$ so it has a known upper bound on its value, then the third inequality is because $p<1$, the equality is our definition of our $y$, then for the next inequality using the fact that we’re assuming that $y_m$ has a particular lower bound since we’re in cases 1 and 2, Then there’s just a cancellation, and $f_1$ only varying by $\delta$ over such tiny distances.

You can use nearly identical arguments in cases 1 and 3 to get that, when you define $z$ to be $z_m-\frac{2\delta }{1-p}$. you have the result that $z\in \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$

Now, in cases 3 and 4, we can let $y$ be: $f_1\left(x\right)-d(f,f^{\prime })$, and then we have:
$-\delta =f_1\left(x\right)-\delta -d(f,f^{\prime })-\left(f_1\right(x)-d(f,f^{\prime }\left)\right)$
$=f_1\left(x\right)-\delta -d(f,f^{\prime })-y\le f_1\left(x_m\right)-d(f,f^{\prime })-y\le y_m-y$
$\le f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta -y\le f_1\left(x\right)+\delta -d(f,f^{\prime })+\frac{2\delta }{p}+\delta -y$
$=f_1\left(x\right)-d(f,f^{\prime })+\frac{2\delta }{p}+2\delta -y=f_1\left(x\right)-d(f,f^{\prime })+\frac{2\delta }{p}+2\delta -\left(f_1\right(x)-d(f,f^{\prime }\left)\right)$
$=\frac{2\delta }{p}+2\delta$
The first equality is just pair-creation, then the second one is packing up the definition of $y$. The first inequality is because $f_1$ only varies by $\delta$ over that distance, the second inequality is because $y_m\in {\psi }^{\prime }\left(x_m\right)$ so it’s got the usual lower bound, then the next inequality after that is because we’re in cases 3 and 4 so
$y_m<f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta$
Then, it’s just another ”$f_1$ doesn’t change much over the tiny distance”, moving the $\delta$’s together, unpacking $y$, and cancelling out. The net result is that we have:
$|y_m-y|\le \frac{2\delta }{p}+2\delta$
You can use nearly identical arguments in cases 2 and 4 to get that, when you define $z$ to be be $f_2\left(x\right)-d(f,f^{\prime })$ you have the result that $|z_m-z|\le \frac{2\delta }{1-p}+2\delta$.

At this point, we can resume our progress on the four cases and go “ok, in case 1, we have...”
$y_m\ge f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta$
$z_m\ge f_2\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{1-p}+\delta$
And we know that those properties lead to $y$ being defined as $y_m-\frac{2\delta }{p}$ and $z$ being defined as $z_m-\frac{2\delta }{p}$. And we know that in that case,
$(y,z)\in \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
So, all we have to check is that $py+(1-p)z\le f^{\prime }\left(x\right)$ in order to conclude that $(y,z)\in {\psi }^{\prime }\left(x\right)$. Let’s do that.
$py+(1-p)z=p(y_m-\frac{2\delta }{p})+(1-p)(z_m-\frac{2\delta }{1-p})=p{y}_m+(1-p)z_m-4\delta$
$\le f^{\prime }\left(x_m\right)-4\delta \le f^{\prime }\left(x\right)+\delta -3\delta =f^{\prime }\left(x\right)-2\delta <f^{\prime }\left(x\right)$
And we have that $(y,z)\in {\psi }^{\prime }\left(x\right)$, accordingly. The first equality was unpacking definitions, then the second was some cancellation, and then the first inequality was because $(y_m,z_m)\in {\psi }^{\prime }\left(x_m\right)$ by assumption so we have $p{y}_m+(1-p)z_m\le f^{\prime }\left(x_m\right)$. The second inequality was because $f^{\prime }$ doesn’t change much over such tiny distances and then it’s just trivial cleanup.

Thus, when we picked a point $(y_m,z_m)\in {\psi }^{\prime }\left(x_m\right)$ where $x_m$ is sufficiently close to $x$, and we’re in case 1, we have that there are points $(y,z)\in {\psi }^{\prime }\left(x\right)$, and
$d\left(\right(y_m,z_m),(y,z\left)\right)=|y_m-y|+|z_m-z|=|y_m-y_m+\frac{2\delta }{p}|+|z_m-z_m+\frac{2\delta }{(1-p)}|$
$=\frac{2\delta }{p}+\frac{2\delta }{(1-p)}$
This is from the definitions of $y$ and $z$ in Case 1.

Now, let’s address case 2, where
$y_m\ge f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta$
$z_m<f_2\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{1-p}+\delta$
In this case, $y$ is defined as $y_m-\frac{2\delta }{p}$, and $z$ is defined as $f_2\left(x\right)-d(f,f^{\prime })$
And we know that in that case,
$(y,z)\in \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
(the first part on the $y$ is the same argument from case 1, the second interval is from the value of $z$)
So, all we have to check is that $py+(1-p)z\le f^{\prime }\left(x\right)$ in order to conclude that $(y,z)\in {\psi }^{\prime }\left(x\right)$. We know that
$-\delta \le z_m-z$
So we can flip this a bit to get
$z\le z_m+\delta$
Accordingly, from that, we get:
$py+(1-p)z\le p(y_m-\frac{2\delta }{p})+(1-p)(z_m+\delta )=p{y}_m+(1-p)z_m-2\delta +(1-p)\delta$
$\le f^{\prime }\left(x_m\right)-\delta \le f^{\prime }\left(x\right)$
And we have that $(y,z)\in {\psi }^{\prime }\left(x\right)$, accordingly.The first inequality was definition unpacking and the inequality we just got, then the first equality is just breaking things up a bit, then the second inequality is just observing that $1-p\le 1$, and then $f^{\prime }$ doesn’t change much over such tiny distances.

Thus, when we picked a point $(y_m,z_m)\in {\psi }^{\prime }\left(x_m\right)$ where $x_m$ is sufficiently close to $x$, and we’re in case 2, we have that there are points $(y,z)\in {\psi }^{\prime }\left(x\right)$, and
$d\left(\right(y_m,z_m),(y,z\left)\right)=|y_m-y|+|z_m-z|$
$\le |y_m-y_m+\frac{2\delta }{p}|+\frac{2\delta }{1-p}+2\delta =\frac{2\delta }{p}+\frac{2\delta }{(1-p)}+2\delta$
This is from the definitions of $y$ and $z$ in Case 2, and the fact that in case 2 we can derive $|z_m-z|\le \frac{2\delta }{1-p}+2\delta$

Extremely similar arguments to case 2 dispatch case 3 with a resolution of the corresponding $(y,z)$ lying in ${\psi }^{\prime }\left(x\right)$ and
$d\left(\right(y_m,z_m),(y,z\left)\right)\le \frac{2\delta }{p}+\frac{2\delta }{(1-p)}+2\delta$

Finally, for case 4, we have:
$y_m<f_1\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{p}+\delta$
$z_m<f_2\left(x_m\right)-d(f,f^{\prime })+\frac{2\delta }{1-p}+\delta$
In this case, $y$ is defined as $f_1\left(x\right)-d(f,f^{\prime })$, and $z$ is defined as $f_2\left(x\right)-d(f,f^{\prime })$
Trivially, we have:
$(y,z)\in \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
So, all we have to check is that $py+(1-p)z\le f^{\prime }\left(x\right)$ in order to conclude that $(y,z)\in {\psi }^{\prime }\left(x\right)$. To do this, we have:
We know that
$-\delta \le z_m-z$
So we can flip this a bit to get
$z\le z_m+\delta$
Accordingly, from that, we get:
$py+(1-p)z=p\left(f_1\right(x)-d(f,f^{\prime })+(1-p\left)\right(f_2\left(x\right)-d(f,f^{\prime }))$
$=p{f}_1\left(x\right)+(1-p)f_2\left(x\right)-d(f,f^{\prime })\le f\left(x\right)-d(f,f^{\prime })\le f^{\prime }\left(x\right)$
(because $p{f}_1+(1-p)f_2\le f$)
And we have that $(y,z)\in {\psi }^{\prime }\left(x\right)$, accordingly.

Thus, when we picked a point $(y_m,z_m)\in {\psi }^{\prime }\left(x_m\right)$ where $x_m$ is sufficiently close to $x$, and we’re in case 4, we have that there are points $(y,z)\in {\psi }^{\prime }\left(x\right)$, and
$d\left(\right(y_m,z_m),(y,z\left)\right)=|y_m-y|+|z_m-z|\le \frac{2\delta }{1-p}+2\delta +\frac{2\delta }{1-p}+2\delta =\frac{2\delta }{p}+\frac{2\delta }{(1-p)}+4\delta$
This is from the definitions of $y$ and $z$ in Case 4, and the fact that in case 4 we can derive $|y_m-y|\le \frac{2\delta }{1-p}+2\delta$ (and same for $z_m$)

These 4 cases were exhaustive, so we now know that, given any $x$ and sequence of points $x_m$ limiting to $x$, and any $\delta$, there is a tail of sufficiently large m’s where the distance from any point in ${\psi }^{\prime }\left(x_m\right)$ to ${\psi }^{\prime }\left(x\right)$ is $\frac{2\delta }{p}+\frac{2\delta }{(1-p)}+4\delta$ or less. We can also flip $x_m$ and $x$ and use our four cases (our argument is symmetric) to show that actually, this is a bound on the Hausdorff distance between ${\psi }^{\prime }\left(x_m\right)$ and ${\psi }^{\prime }\left(x\right)$. $\delta$ was arbitrary, as was the sequence $x_m$ and the $x$, so this means that ${\psi }^{\prime }$ is continuous in Hausdorff-distance.

Ok, we’re a bit in the weeds here, how does that help? Well, we were trying to verify the compact almost-support property for the supremum. This requires, as part of it, getting a continuous function with some special properties. We’re going to apply a selection function to get it, but we could only take care of the prerequisites that aren’t lower-hemicontinuity. And to show lower-Hemicontinuity in general, we needed to take this detour through showing that the modified set-valued function is continuous in Hausdorff-distance. So let’s pop back up the stack.

One level back up the stack, we were trying to show lower-Hemicontinuity. It is the property that given any sequence $x_n$ which limits to $x$, and any $(y,z)\in \psi \left(x\right)$, there is some subsequence $x_m$ and $(y_m,z_m)\in \psi \left(x_m\right)$ where $(y_m,z_m)$ limits to $\psi \left(x_m\right)$. We dispatched the case where infinitely many elements of the sequence were in our $C_{ϵ}^1\cup C_{ϵ}^2$, leaving two cases. There’s the case where only finitely elements of that sequence are in that compact set, but the limit point $x$ lies in that set. There’s also the case where the limit point $x$ doesn’t lie in that set.

Dealing with case 3, we have a sequence $x_n$ heading to $x$. Strip off all the $x_n$ that lie in the compact set, making your $x_m$. And let $(y_m,z_m)$ be whichever point in $\psi \left(x_m\right)$ is closest to $(y,z)$. Now, by how they were defined, $\psi \left(x_m\right)={\psi }^{\prime }\left(x_m\right)$, and ${\psi }^{\prime }$ is continuous in Hausdorff-distance, so “take the closest point” is definitely going to get you the convergence you seek to your arbitrarily selected $(y,z)\in \psi \left(x\right)$ point.

For case 2, where we’re limiting to $x$ from outside the compact set, all we need to show is that $\psi \left(x\right)\subseteq {\psi }^{\prime }\left(x\right)$ (we don’t necessarily have equality because $\psi$ and ${\psi }^{\prime }$ start being different on that compact set), in order to get a sequence $(y_m,z_m)$ converging to the $(y,z)\in \psi \left(x\right)$ point. So, let’s do this. Because $x$ lies in $C_{ϵ}^1\cup C_{ϵ}^2$, we have that $\psi \left(x\right)=\left(f_1\right(x),f_2(x\left)\right)$.
The conditions for $(y,z)$ to be in ${\psi }^{\prime }\left(x\right)$ are:
$(y,z)\in \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
Which is obviously true for $f_1\left(x\right),f_2\left(x\right)$, and:
$py+(1-p)z\le f^{\prime }\left(x\right)$
Which is the case because:
$p{f}_1\left(x\right)+(1-p)f_2\left(x\right)\le f\left(x\right)=f^{\prime }\left(x\right)$
By how $f$ was made, and $f^{\prime }=f$ on that compact set.

Thus, we’re done, we verified lower-hemicontinuity for $\psi$ in all the cases, so we can invoke the Michael selection theorem and get a continuous selection $f^{\ast }:X\to R^2$ with three valuable properties. Let’s abbreviate $p{r}_1\left(f^{\ast }\right(x\left)\right)$ as $f_1^{\prime }$, for notational convenience. It’s projecting it to the first coordinate.$f_2^{\prime }$ is defined similarly.

Our first notable property is:
$x\in C_{ϵ}^1\cup C_{ϵ}^2\to f_1^{\prime }\left(x\right)=f_1\left(x\right)\wedge f_2^{\prime }\left(x\right)=f_2\left(x\right)$
(ie, projecting $f^{\ast }$ down to the two coordinates makes functions which perfectly mimic $f_1$ and $f_2$ on the compact set of interest)

Our second one is:
$d(f_1,f_1^{\prime })\le d(f,f^{\prime })$
And the same for $d(f_2,f_2^{\prime })$.

And our third notable property is that:
$p{f}_1^{\prime }+(1-p)f_2^{\prime }\le f^{\prime }$
But why do these properties hold of our selection function? Well, when $x$ lies in that compact set, $\psi \left(x\right)=\left(f_1\right(x),f_2(x\left)\right)$, so our selection function is forced to have its projections mimic $f_1$ and $f_2$ on said compact set, taking care of the first one.

For our second property, we have:
$f^{\ast }\left(x\right)\in \psi \left(x\right)\subseteq \left[f_1\right(x)-d(f,f^{\prime }),f_1(x)+d(f,f^{\prime }\left)\right]\times \left[f_2\right(x)-d(f,f^{\prime }),f_2(x)+d(f,f^{\prime }\left)\right]$
Accordingly, we know that the projections to the two coordinates can’t be too far away from $f_1$and $f_2$ respectively.

For our third property, we have:
$f^{\ast }\left(x\right)\in \psi \left(x\right)\subseteq \left\{\right(y,z)|py+(1-p)z\le f^{\prime }(x\left)\right\}$
Accordingly, the projections to the two coordinates can’t mix to exceed the function $f^{\prime }$.

So, where to from here? Well, we have:
$|sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)|=sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
$\simeq p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)-sup(h_1,h_2)\left(f^{\prime }\right)$
$\le p\left(h_1\right(f_1^{\prime })+ϵd(f,f^{\prime }\left)\right)+(1-p)h_2\left(f_2^{\prime }\right)+ϵd(f,f^{\prime }))-sup(h_1,h_2\left)\right(f^{\prime })$
$=p{h}_1\left(f_1^{\prime }\right)+(1-p)h_2\left(f_2^{\prime }\right)-sup(h_1,h_2)\left(f^{\prime }\right)+ϵd(f,f^{\prime })\le ϵd(f,f^{\prime })$
Here’s why. We assumed at the very start that without loss of generality, we’d take $f$ to be the one with higher expectation value. We found a $p,f_1,f_2$ that nearly replicated the expectation value of $sup(h_1,h_2)\left(f\right)$. $f_1^{\prime }$ copies $f_1$ on a compact almost-support of $h_1$, namely $C_{ϵ}^1$, and we also have $d(f_1^{\prime },f_1)\le d(f^{\prime },f)$, and similar for $f_2^{\prime }$ and $f_2$. And finally, since $p{f}_1^{\prime }+(1-p)f_2^{\prime }\le f^{\prime }$, that mix must have lower value than $sup(h_1,h_2)\left(f^{\prime }\right)$. And we’re done!$f$ and $f^{\prime }$ were arbitrary except that they agreed on $C_{ϵ}^1\cup C_{ϵ}^2$, a compact set, and we got:
$sup(h_1,h_2)\left(f\right)-sup(h_1,h_2)\left(f^{\prime }\right)\le ϵd(f,f^{\prime })$
Witnessing that said set is a compact $ϵ$-almost-support.$ϵ$ was arbitrary, so $sup(h_1,h_2)$ is compactly-almost-supported. This is the last condition needed to check to see that it’s an infradistribution.

Proposition 15: All three characterizations of the supremum given in Definition 13 are identical.

So, the first characterization we gave was:
$sup(h_1,h_2)\left(f\right):={sup}_{p,f_1,f_2:p{f}_1+(1-p)f_2\le f}p{h}_1\left(f_1\right)+(1-p)h_2\left(f_2\right)$
And the second characterization was the least infradistribution greater than $h_1,h_2$ in the information ordering.

And the third characterization was as the concave monotone hull of $f\mapsto sup\left(h_1\right(f),h_2(f\left)\right)$.

We will use ${sup}_1,{sup}_2,{sup}_3$ for these three characterizations of the supremum of two infradistributions and show that they are equal.

Let’s begin showing this.
${sup}_2(h_1,h_2)\left(f\right)\ge {sup}_{\zeta \in \Delta N,f_i\in {\prod }_i{C}_B\left(X\right):E_{\zeta }{f}_i\le f}\left({sup}_2\right(h_1,h_2\left)\right(E_{\zeta }\left(f_i\right)\left)\right)$
This occurs by monotonicity, any mix of functions which undershoots $f$ must get a lower score because ${sup}_2(h_1,h_2)$ is an infradistribution.
$\ge {sup}_{\zeta \in \Delta N,f_i\in {\prod }_i{C}_B\left(X\right):E_{\zeta }{f}_i\le f}{E}_{\zeta }\left({sup}_2\right(h_1,h_2\left)\right(f\left)\right)$
This is because of convexity of ${sup}_2(h_1,h_2)$, since it’s an infradistribution. The value of the mix is as good or better than the mix of the values.
$\ge {sup}_{\zeta \in \Delta N,f_i\in {\prod }_i{C}_B\left(X\right):E_{\zeta }{f}_i\le f}{E}_{\zeta }(sup(h_1\left(f_i\right),h_2\left(f_i\right)\left)\right)={sup}_3(h_1,h_2)\left(f\right)$
This is because ${sup}_2(h_1,h_2)\ge h_1$ (and same for $h_2$), so making that swap decreases the value. Also, this quantity is the concave monotone hull of the supremum of $h_1,h_2$. Why? Well, $sup\left(h_1\right(f),h_2(f\left)\right)$ is our first attempt at assessing the value of a function $f$. However, it isn’t necessarily monotone. So, ${sup}_{f^{\ast }\le f}sup\left(h_1\right(f^{\ast }),h_2(f^{\ast }\left)\right)$ is the monotone hull, we’re saying that if there’s a value below you that outscores you, then you should update the value of $f$ to be big enough. And then, to get the concave monotone hull, we replace the lower bound on $f$ with a countable/​arbitrary finite mix of functions because any concave function should have the value of the mix be $\ge$ the mix of the values, so we have to bump the value of $f^{\ast }$ up to at least the mix of the values to not violate concavity. Anyways, now that we know this is ${sup}_3(h_1,h_2)\left(f\right)$, we can go further to:
$\ge {sup}_{p,f_1,f_2:p{f}_1+(1-p)f_2\le f}(psup(h_1\left(f_1\right),h_2\left(f_1\right))+(1-p)sup(h_1\left(f_2\right),h_2\left(f_2\right)\left)\right)$
This is lower because now we’re specializing to only certain sorts of probability distributions over $N$, those that are only supported on the first two values, so it’s harder to attain suprema. And now,
$\ge {sup}_{p,f_1,f_2:p{f}_1+(1-p{)}_2\le f}\left(p{h}_1\right(f_1)+(1-p\left)h_2\right(f_2\left)\right)={sup}_1(h_1,h_2)\left(f\right)$
We swapped out the supremum for a specific term in it in order to do this, and used our given definition of ${sup}_1$. And then we can specialize $p$ to 1 and $f_1$ to $f$ itself, to get
$\ge h_1\left(f\right)$
Similarly, we could specialize to $p=0$ and $f_2=f$ to get $\ge h_2\left(f\right)$. So taking stock of what we have,
${sup}_2(h_1,h_2)\left(f\right)\ge {sup}_3(h_1,h_2)\left(f\right)\ge {sup}_1(h_1,h_2)\left(f\right)\ge sup\left(h_1\right(f),h_2(f\left)\right)$
For all functions, so:
${sup}_2(h_1,h_2)\ge {sup}_3(h_1,h_2)\ge {sup}_1(h_1,h_2)\ge h_1$
(and same for $h_2$) We recall that in Proposition 14 we proved that ${sup}_1$ always makes an infradistribution. Since ${sup}_1$ is above both component infradistributions, and ${sup}_2$ was defined as the least infradistribution that is above $h_1$ and $h_2$, we must have equality, and
${sup}_2(h_1,h_2)={sup}_3(h_1,h_2)={sup}_1(h_1,h_2)\ge h_1$
(and same for $h_2$) And we’ve shown the three definitions of the supremum are identical.

Proposition 16:
$E_{sup(H_1,H_2)}\left(f\right)={sup}_{p,f_1,f_2:p{f}_1+(1-p)f_2\le f}p{E}_{H_1}\left(f_1\right)+(1-p)E_{H_2}\left(f_2\right)$

To recap,
$sup(H_1,H_2):=H_1\cap H_2$
Now, $sup(H_1,H_2)$ can be turned into a concave monotone functional $C_B\left(X\right)\to R$, by LF-duality. Further, it’s convex, closed, and upper-complete due to being the intersection of two convex closed upper-complete sets. Let’s use $h$ to refer to its corresponding functional. Then:
$h\left(f\right)=E_{sup(H_1,H_2)}\left(f\right)={inf}_{(m,b)\in H_1\cap H_2}m\left(f\right)+b$
$\ge {inf}_{(m,b)\in H_1}m\left(f\right)+b=E_{H_1}\left(f\right)=h_1\left(f\right)$
And the same applies to $H_2$, and this applies to all functions, so $h\ge h_1$ (and same for $h_2$).

We know from Proposition 15 that the least concave monotone functional above $h_1$ and $h_2$ is $sup(h_1,h_2)$, so $h\ge sup(h_1,h_2)\ge h_1$ (and same for $h_2$) Call the corresponding set of $sup(h_1,h_2)$ as $H_{sup}$. Thus, translating this information ordering back to sets,
$sup(H_1,H_2)\subseteq H_{sup}\subseteq H_1$
And same for $H_2$. Therefore.
$sup(H_1,H_2)\subseteq H_{sup}\subseteq H_1\cap H_2=sup(H_1,H_2)$
Therefore, all the subsets must be actual equalities, and so in particular we have:
$H_{sup}=sup(H_1,H_2)$
Then we can go:
$E_{sup(H_1,H_2)}\left(f\right)=sup(h_1,h_2)\left(f\right)$
$={sup}_{p,f_1,f_2}\left(p{h}_1\right(f_1)+(1-p\left)h_2\right(f_2\left)\right)={sup}_{p,f_1,f_2:p{f}_1+(1-p)f_2\le f}\left(p{E}_{H_1}\right(f_1)+(1-p\left)E_{H_2}\right(f_2\left)\right)$
By $sup(H_1,H_2)$ being equivalent to the infradistribution set induced by $sup(h_1,h_2)$, expanding our definition of the sup, and translating back. And we’re done!

Proposition 17: For any property in the table at the start of this section, $sup(h_1,h_2)$ will fulfill the property if both components fulfill the property.

The way to show this is to use the alternate characterizations of supremum as intersection of the infradistribution sets, and the alternate characterizations of the various properties in terms of properties of minimal points.

We will make an observation used in all further proofs of properties. In order for $H_1$ to have $(\lambda \mu ,b)$ in it, there must be a minimal point of the form $(\lambda \mu ,b_1)$ with $b_1\le b$ below it. Similarly, for $H_2$ to contain $(\lambda \mu ,b)$, there must be a minimal point of the form $(\lambda \mu ,b_2)$ below it, with $b_2\le b$.

Thus, for $(\lambda \mu ,b)$ to lie in $(H_1\cap H_2{)}^{min}$, $(\lambda \mu ,b_1)\in H_1^{min}$ and $(\lambda \mu ,b_2)\in H_2^{min}$ and $b=sup(b_1,b_2)$. Part of this is because said point lies in $H_1$ and $H_2$, the other part is because $(\lambda \mu ,sup(b_1,b_2\left)\right)$ is the lowest possible point in $H_1\cap H_2$ associated with a measure component of $\lambda \mu$, and it’s the minimal. This observation will be used for all future sub-proofs in this proposition.

Homogenity: This is equivalent to “all minimal points have $b=0$”, so if $(\lambda \mu ,b)\in (H_1\cap H_2{)}^{min}$, then $(\lambda \mu ,0)\in H_1^{min}$ (homogenity for $H_1$), and same for $H_2^{min}$, so $b=sup(0,0)=0$.

1-Lipschitzness: This is equivalent to “all minimal points have $\lambda \le 1$”, so if $(\lambda \mu ,b)\in (H_1\cap H_2{)}^{min}$, then $(\lambda \mu ,b_1)\in H_1^{min}$, and $\lambda \le 1$ (1-Lipschitzness of $H_1$), so $\lambda \le 1$.

Cohomogenity: This is equivalent to “all minimal points have $\lambda +b=1$”, so if $(\lambda \mu ,b)\in (H_1\cap H_2{)}^{min}$, then $(\lambda \mu ,b_1)\in H_1^{min}$, and $\lambda +b_1=1$ (cohomogenity of $H_1$), and $(\lambda \mu ,b_2)\in H_2^{min}$, and $\lambda +b_2=1$, so $b_1=b_2$. Then, $\lambda +b=\lambda +sup(b_1,b_2)=\lambda +b_1=1$.

C-additivity: This is equivalent to “all minimal points have $\lambda =1$”, so if $(\lambda \mu ,b)\in (H_1\cap H_2{)}^{min}$, then $(\lambda \mu ,b_1)\in H_1^{min}$, and $\lambda =1$ (C-additivity of $H_1$), so $\lambda =1$.

Crispness: This is equivalent to the conjunction of homogenity and C-additivity, both of which are preserved, so crispness is preserved as well.

Sharpness: Because all sharp infradistributions are crisp, $(H_1\cap H_2{)}^{min}$ must be composed entirely of probability distributions if $H_1$ and $H_2$ are sharp. If any of the probability distributions in $(H_1\cap H_2{)}^{min}$ aren’t supported on $C_1$ (the compact set associated with the sharp infradistribution $H_1$), then they aren’t in $H_1$, which is impossible. Symmetric arguments apply to $H_2$. Thus, $(H_1\cap H_2{)}^{min}$ only has probability distributions supported on $C_1\cap C_2$. If there was any probability distribution supported on that set that was missing from $(H_1\cap H_2{)}^{min}$, then it’d be present in $H_1$ and $H_2$, and thus present in $H_1\cap H_2$, and minimal, so we have a contradiction. Therefore $(H_1\cap H_2{)}^{min}$ consists of all probability distributions supported on $C_1\cap C_2$ which is a compact set, so the supremum is sharp as well.

Proposition 18: If a family of infradistributions $h_i$ is directifiable, then ${sup}_i{h}_i$ (defined as the functional corresponding to the set ${\bigcap }_i{H}_i$) exists and is an infradistribution. Further, for all conditions listed in the table, if all the $h_i$ fulfill them, then ${sup}_i{h}_i$ fulfills the same property.

A family of infradistributions being directifiable is equivalent to “for any collection of finitely many infradistributions, the supremum exists”. We also know that the supremum is exactly equivalent to set intersection. So, we’ll show that directifiability (any collection of finitely many infradistributions has a supremum) implies that the intersection of all the infradistribution sets has the exact properties of a set-form infradistribution.

We have six properties to check. Nonemptiness, normalization (the existence of a point $(\lambda \mu ,0)$, existence of a point $(\lambda \mu ,b)$ with $\lambda +b=1$ and nonexistence of points with $\lambda +b<1$), closure, convexity, upper-completion, and compact-projection (the measure components of the infradistribution are contained in a compact set of measures).

For closure, it’s the intersection of closed sets, so it’s closed. For convexity, it’s the intersection of convex sets, so it’s convex. For upper-completion, it’s the intersection of upper-complete sets, so it’s upper-complete. For compact-projection, the measure components of the countable intersection are contained within the countable intersection of the sets of measure components, which is contained in a compact set, so it fulfills that property too.

This just leaves nonemptiness and normalization. We’ll show normalization, which automatically implies nonemptiness. The nonexistence of points with $\lambda +b<1$ is definitely not preserved under intersection.

However, the compact-projection property means that for any infradistribution set $H_i$, the intersection of it with the surface of a-measures where $\lambda +b=1$ is compact, so we’re intersecting a bunch of compact sets. Due to the existence of supremum infradistributions for each collection of finitely many infradistributions (directifiability), we have the nonempty finite intersection property needed to conclude that the intersection of compact sets is nonempty. The same argument applies to the existence of a point with $b=0$. The presence of those two points witnesses nonemptiness and normalization.

These are the last two conditions we needed to conclude the set represents an infradistribution, so the infinite supremum exists and is the infradistribution we need.

For preservation of the various properties, we can just reuse the arguments from Proposition 17 with only trivial modifications.