Although this post is old, I really dislike the description of confidence levels. An appropriate confidence level is made out of the points with the “best” likelihoods: Every point in the point has a higher likelihood than every point outside. Therefore, an output of ‘[“Cheesecake”-”Cheddar”]’ is a clear sign of bad science.
A good confidence level is just a summary of likelihood ratios. And for symmetric likelihood profiles (at least near the minimum), it gives a very good idea of the function in the interesting range.
And most problems with p-values can be reduced by choosing the study and analysis technique in advance (or with blinded data, for the analysis part). If you can flip the coin again, make a small number of hypotheses and test them. Writing down good prior probabilities after looking at coin tosses does not really work with human brains, and writing down 100000 different options (including things like “the coin gives always the repeating pattern TTTTTH”) before doing coin tosses is even worse.
Although this post is old, I really dislike the description of confidence levels. An appropriate confidence level is made out of the points with the “best” likelihoods: Every point in the point has a higher likelihood than every point outside. Therefore, an output of ‘[“Cheesecake”-”Cheddar”]’ is a clear sign of bad science. A good confidence level is just a summary of likelihood ratios. And for symmetric likelihood profiles (at least near the minimum), it gives a very good idea of the function in the interesting range.
And most problems with p-values can be reduced by choosing the study and analysis technique in advance (or with blinded data, for the analysis part). If you can flip the coin again, make a small number of hypotheses and test them. Writing down good prior probabilities after looking at coin tosses does not really work with human brains, and writing down 100000 different options (including things like “the coin gives always the repeating pattern TTTTTH”) before doing coin tosses is even worse.