The noise ε isn’t independent of the context—it is chosen to be noise that produces the context when fed into the predictor’s model. So you can reconstruct the situation by rerunning select parts of the predictor. But doing so is then expensive, which encourages you to rerun as little of the predictor as possible.
Probably I’m wrong because it does not seem like the reporter function should have access to before and action. But I don’t see how you can reconstruct the context when you cannot calculate the probability distribution from which to sample using the noise.
I would suspect that the code in some way doesn’t reflect your thinking. Currently you could replace the line
without changing anything. My guess is that either this should not be that way or that the reporter should have access to before and action.
(But anyway, I think the overall approach is quite clear. I just don’t get your implementation and would implement it differently, so feel free not to bother much about it.)
You’re right that I’ve done something wrong. I was thinking about predictor being a generative model for everything starting from scratch (this is a setting I often consider), so that you can use ε to reconstruct everything. But the way it’s currently set up in the doc, the predictor is a generative model only for after. So we need to give before and action.
The noise ε isn’t independent of the context—it is chosen to be noise that produces the context when fed into the predictor’s model. So you can reconstruct the situation by rerunning select parts of the predictor. But doing so is then expensive, which encourages you to rerun as little of the predictor as possible.
Sorry, I’m still not sure I got it.
Is it that in the reporter(question, eps, theta_reporter) function you can call the predictor like something like that:
z_part = predict_part(part, before, action, theta).sample_using_noise(eps)
Probably I’m wrong because it does not seem like the reporter function should have access to before and action. But I don’t see how you can reconstruct the context when you cannot calculate the probability distribution from which to sample using the noise.
I would suspect that the code in some way doesn’t reflect your thinking. Currently you could replace the line
z, ε = posterior(before, action, after, θ).sample_with_noise()
from the reporter_loss function with the line
ε = gaussian.sample()
without changing anything. My guess is that either this should not be that way or that the reporter should have access to before and action.
(But anyway, I think the overall approach is quite clear. I just don’t get your implementation and would implement it differently, so feel free not to bother much about it.)
You’re right that I’ve done something wrong. I was thinking about predictor being a generative model for everything starting from scratch (this is a setting I often consider), so that you can use ε to reconstruct everything. But the way it’s currently set up in the doc, the predictor is a generative model only for after. So we need to give before and action.