scottviteri comments on The Geometric Expectation

[scottviteri]

I think what is going on here is that both ${\nabla }^{\ast }$ and $G$ are of the form $\left(e^{\wedge }\right)\circ g\circ \ln$ with $g=\nabla$ and $g=E$, respectively. Let’s define the star operator as $g^{\ast }=\left(e^{\wedge }\right)\circ g\circ \ln$. Then $(f\circ g{)}^{\ast }=(e^{\wedge })\circ (f\circ g)\circ \ln =(e^{\wedge })\circ f\circ \ln \circ (e^{\wedge })\circ g\circ \ln =f^{\ast }\circ g^{\ast }$, by associativity of function composition. Further, if $f$ and $g$ commute, then so do $f^{\ast }$ and $g^{\ast }$: $g^{\ast }\circ f^{\ast }=(g\circ f{)}^{\ast }=(f\circ g{)}^{\ast }=f^{\ast }\circ g^{\ast }.$

So the commutativity of the geometric expectation and derivative fall directly out of their representation as $E^{\ast }$ and ${\nabla }^{\ast }$, respectively, by commutativity of $E$ and $\nabla$, as long as they are over different variables.

We can also derive what happens when the expectation and gradient are over the same variables: $({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}{)}^{\ast }$. First, notice that $(\ast k{)}^{\ast }(x)=e^{k\ast \ln x}=e^{\ln x\ast k}=x^k$, so $(\ast k{)}^{\ast }={(}^{\wedge }k)$.. Also $(+k{)}^{\ast }(x)=e^{k+\ln \left(x\right)}=e^k{e}^{\ln \left(x\right)}=x{e}^k⟹(+k{)}^{\ast }=(\ast e^k)$.

Now let’s expand the composition of the gradient and expectation. $({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)})\left(f\right(x\left)\right)={\nabla }_{\theta }\int P_{\theta }\left(x\right)f\left(x\right)dx=E_{x\sim P_{\theta }\left(x\right)}\left[{\nabla }_{\theta }\right(f\left(x\right)\ln P_{\theta }\left(x\right)\left)\right]$, using the log-derivative trick. So ${\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}=E_{x\sim P_{\theta }\left(x\right)}\circ {\nabla }_{\theta }\circ (\ast \ln P_{\theta }(x\left)\right)$.

Therefore, ${\nabla }_{\theta }^{\ast }\circ G_{x\sim P_{\theta }\left(x\right)}=({\nabla }_{\theta }\circ E_{x\sim P_{\theta }\left(x\right)}{)}^{\ast }$ $=E_{x\sim P_{\theta }\left(x\right)}^{\ast }\circ {\nabla }_{\theta }^{\ast }\circ (\ast \ln P_{\theta }(x){)}^{\ast }$ $=G_{x\sim P_{\theta }}\circ {\nabla }_{\theta }^{\ast }\circ {(}^{\wedge }\ln P_{\theta })$.

Writing it out, we have ${\nabla }_{\theta }^{\ast }{G}_{x\sim P_{\theta }\left(x\right)}\left[f\right(x\left)\right]=G_{x\sim P_{\theta }\left(x\right)}\left[{\nabla }_{\theta }^{\ast }\right(f\left(x{)}^{\ln P_{\theta }\left(x\right)}\right]$.