What’s interesting to me about this process is that it almost tries to avoid induction altogether. Only the move from step 4 to 5 seems anything like an inductive argument. The rest is purely deductive—though admittedly it takes a couple premises in order to quantify just how likely our sample was and that surely has something to do with induction. But it’s still a bit like solving the problem of induction by sweeping it under the rug then putting a big heavy deduction table on top so no one notices the lumps underneath.
One interesting thing here is that you start with a null vs other hypothesis, but that’s because you’re doing a two-sample z/t-test. But what’s going on when you do a one sample z/t-test and get out a confidence interval?
The one sided hypothesis test is still null vs. other because it uses the full parameter space, i.e. it’s H0: mu ⇐ c vs. Ha: mu > c. We present it to undergrads as H0: mu = c vs. Ha: mu > c in order to simplify (I think that’s the reason anyway) but really we’re testing the former. The Karlin-Rubin theorem justifies this.
I don’t follow… that sounds like you’re giving the definition of a one-tailed hypothesis test. What does that have to do with a constant c? Suppose I do this in R:
R> set.seed(12345); t.test(rnorm(20))
One Sample t-test
data: rnorm(20)
t = 0.4103, df = 19, p-value = 0.6861
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
-0.3138 0.4668
sample estimates:
mean of x
0.07652
And get a 95% CI of (-0.3138-0.4668); if my null hypothesis (H0) is my mu or sample mean (0.07652), then you say my Ha is mu > c, or 0.07652 > c. What is this c?
So rereading your first comment, I realize you said one-sample vs. two-sample hypothesis test and not one-sided vs. two-sided (ore one-tailed vs. two-tailed). If that’s what you meant, I don’t follow your first comment. The t-test I gave in the post is a one-sample test—and I don’t understand how the difference between the two is relevant here.
But to answer your question anyway:
I don’t follow… that sounds like you’re giving the definition of a one-tailed hypothesis test. What does that have to do with a constant c? Suppose I do this in R:
And get a 95% CI of (-0.3138-0.4668); if my null hypothesis (H0) is my mu or sample mean (0.07652), then you say my Ha is mu > c, or 0.07652 > c. What is this c?
c is the value you’re testing as the null hypothesis. In that R-code, R assumes that c=0 so that H0: mu=c and Ha: mu=/=c. For the R code:
I’m interested in the calculated confidence interval, not the p-value necessarily. Noodling around some more, I think I’m starting to understand it more: the confidence interval isn’t calculated with respect to the H0 of 0 which the R code defaults to, it’s calculated based purely on the mean (and then an H0 of 0 is assumed to spit out some p-value)
R> set.seed(12345); t.test(rnorm(20,100,15))
One Sample t-test
data: rnorm(20, 100, 15)
t = 36.16, df = 19, p-value < 2.2e-16
alternative hypothesis: true mean is not equal to 0
95 percent confidence interval:
95.29 107.00
sample estimates:
mean of x
101.1
R>
R> 107-95.29
[1] 11.71
R> 107 - (11.71/2)
[1] 101.1
Hm… I’m trying to fit this assumption into your framework....
Either h0, true mean = sample mean; or ha, true mean != sample mean
construct the test statistic: ‘t = sample mean—sample mean / s/sqrt(n)’
A 95% confidence interval is sort of like testing H0:mu=c vs Ha:mu=\=c for all values of c at the same time. In fact if you reject the null hypothesis for a given c when c is outside your calculated confidence interval and fail to reject otherwise, you’re performing the exact same t-test with the exact same rejection criteria as the usual one (that is if the p-value is less than 0.05).
The formula for the test statistic is (generally) t = (estimate—c)/(standard error of estimate) while the formula for a confidence interval is (generally) estimate +/- t^(standard error of estimate) where t^ is a quantile of the t distribution with appropriate degrees of freedom, chosen according to your desired confidence level. t^* and the threshold for rejecting the null in a hypothesis test are intimately related. If you google “confidence intervals and p values” I’m sure you’ll find a more polished and detailed explanation of this than mine.
One interesting thing here is that you start with a null vs other hypothesis, but that’s because you’re doing a two-sample z/t-test. But what’s going on when you do a one sample z/t-test and get out a confidence interval?
The one sided hypothesis test is still null vs. other because it uses the full parameter space, i.e. it’s H0: mu ⇐ c vs. Ha: mu > c. We present it to undergrads as H0: mu = c vs. Ha: mu > c in order to simplify (I think that’s the reason anyway) but really we’re testing the former. The Karlin-Rubin theorem justifies this.
I don’t follow… that sounds like you’re giving the definition of a one-tailed hypothesis test. What does that have to do with a constant c? Suppose I do this in R:
And get a 95% CI of (-0.3138-0.4668); if my null hypothesis (H0) is my mu or sample mean (0.07652), then you say my Ha is mu > c, or 0.07652 > c. What is this c?
So rereading your first comment, I realize you said one-sample vs. two-sample hypothesis test and not one-sided vs. two-sided (ore one-tailed vs. two-tailed). If that’s what you meant, I don’t follow your first comment. The t-test I gave in the post is a one-sample test—and I don’t understand how the difference between the two is relevant here.
But to answer your question anyway:
c is the value you’re testing as the null hypothesis. In that R-code, R assumes that c=0 so that H0: mu=c and Ha: mu=/=c. For the R code:
You perform a t test with H0: mu<=c and Ha: mu>c.
I’m interested in the calculated confidence interval, not the p-value necessarily. Noodling around some more, I think I’m starting to understand it more: the confidence interval isn’t calculated with respect to the H0 of 0 which the R code defaults to, it’s calculated based purely on the mean (and then an H0 of 0 is assumed to spit out some p-value)
Hm… I’m trying to fit this assumption into your framework....
Either h0, true mean = sample mean; or ha, true mean != sample mean
construct the test statistic: ‘t = sample mean—sample mean / s/sqrt(n)’
‘t = 0 / s/sqrt(n)’; t = 0
… a confidence interval
A 95% confidence interval is sort of like testing H0:mu=c vs Ha:mu=\=c for all values of c at the same time. In fact if you reject the null hypothesis for a given c when c is outside your calculated confidence interval and fail to reject otherwise, you’re performing the exact same t-test with the exact same rejection criteria as the usual one (that is if the p-value is less than 0.05).
The formula for the test statistic is (generally) t = (estimate—c)/(standard error of estimate) while the formula for a confidence interval is (generally) estimate +/- t^(standard error of estimate) where t^ is a quantile of the t distribution with appropriate degrees of freedom, chosen according to your desired confidence level. t^* and the threshold for rejecting the null in a hypothesis test are intimately related. If you google “confidence intervals and p values” I’m sure you’ll find a more polished and detailed explanation of this than mine.