Think I have finally got it. I would like to thank you once again for all your help; I really appreciate it.

This is what I think “estimating the probability” means:

We define theta to be a real-world, objective, physical parameter/quantity s.t. P(H|theta=alpha) = alpha & P(T|theta=alpha) = 1 - alpha. We do not talk about the nature of this quantity theta because we do not care what it is. I don’t think it is appropriate to say that theta is “frequency” for this reason:

“frequency” is not a well-defined physical quantity. You can’t measure “frequency” like you measure temperature.

But we do not need to dispute about this as theta being “frequency” is unnecessary.

Using the above definitions, we can compute the likelihood and then the posterior and then the posterior predictive which is represents the probability of heads given data from previous flips.

Is the above accurate?

So Bayesians who say that theta is the probability of heads and compute a point estimate of the parameter theta and say that they have “estimated the probability” are just frequentists in disguise?

No worries :) Thanks a lot for your help! Much appreciated.

It’s amazing how complex a simple coin flipping problem can get when we approach it from our paradigm of objective Bayesianism. Professor Jaynes remarks on this after deriving the principle of indifference: “At this point, depending on your personality and background in this subject, you will be either greatly impressed or greatly disappointed by the result (2.91).”—page 40

A frequentist would have “solved“ this problem rather easily. Personally, I would trade simplicity for coherence any day of the week...