And the latter is usually what we actually use in basic analysis of experimental data—e.g. to decide whether there’s a significant different between the champagne-drinking group and the non-champagne-drinking group
I never bought up null-hypothesis testing in the liver weight example and it was not meant to illustrate that… hence why I never bought up the idea of signfiance.
Mind you, I disagree that signficance testing is done correctly, but this is not the argument against it nor is it related to it.
(The OP also complains that “We can’t determine the interval for which most processes will yield values”. This is not necessarily a problem; there’s like a gazillion versions of the CLT, and not all of them depend on bounding possible values. CLT for e.g. the Cauchy distribution even works for infinite variance.)
My argument is not that you can’t come up with a distribution for every little edge case imaginable, my argument is exactly that you CAN and you SHOULD but this process should be done automatically, because every single problem is different and we have the means to dynamically see the model that best suits every problem rather than stick to choosing between e.g. 60 names distributions.
Even here, we can apply a linearity → normality argument as long as the errors are small relative to curvature.
I fail to see your argument here, as in, I fail to see how it deals with the interconnected bit of my argument and I fail to see how noise being small is something that ever happens in a real system, in the sense you use it here, as in, noise being everything that’s not inference we are looking for.
There absolutely is a property of mathematics that tells us what a slightly-off right-angled triangle is: it’s a triangle which satisfies Pythagoras’ formula, to within some uncertainty.
But, by this definition that you use here, any arbitrary thing I want to define mathematically, even if it contains within it some amount of hand wavyness or uncertainty, can be a property of mathematics ?
Your article seems to have some assumption that increase complexity == proneness to overfitting.
Which in itself is true if you aren’t validating the model, but if you aren’t validating the model it seems to me that you’re not even in the correct game.
If you are validating the model, I don’t see how the argument holds (will look into the book tomorrow if I have time)
Intuitively, it’s the same idea as conservation of expected evidence: if one model predicts “it will definitely be sunny tomorrow” and another model predicts “it might be sunny or it might rain”, and it turns out to be sunny, then we must update in favor of the first model. In general, when a complex model is consistent with more possible datasets than a simple model, if we see a dataset which is consistent with the simple model, then we must update in favor of the simple model. It’s that simple. Bayesian model comparison quantifies that idea, and gives a more precise tradeoff between quality-of-fit and model complexity.
I fail to understand this argument and I did previously read the article mentioned here, but maybe it’s just a function of it being 1AM here, I will try again tomorrow.
I never bought up null-hypothesis testing in the liver weight example and it was not meant to illustrate that… hence why I never bought up the idea of signfiance.
Mind you, I disagree that signficance testing is done correctly, but this is not the argument against it nor is it related to it.
My argument is not that you can’t come up with a distribution for every little edge case imaginable, my argument is exactly that you CAN and you SHOULD but this process should be done automatically, because every single problem is different and we have the means to dynamically see the model that best suits every problem rather than stick to choosing between e.g. 60 names distributions.
I fail to see your argument here, as in, I fail to see how it deals with the interconnected bit of my argument and I fail to see how noise being small is something that ever happens in a real system, in the sense you use it here, as in, noise being everything that’s not inference we are looking for.
But, by this definition that you use here, any arbitrary thing I want to define mathematically, even if it contains within it some amount of hand wavyness or uncertainty, can be a property of mathematics ?
Your article seems to have some assumption that increase complexity == proneness to overfitting.
Which in itself is true if you aren’t validating the model, but if you aren’t validating the model it seems to me that you’re not even in the correct game.
If you are validating the model, I don’t see how the argument holds (will look into the book tomorrow if I have time)
I fail to understand this argument and I did previously read the article mentioned here, but maybe it’s just a function of it being 1AM here, I will try again tomorrow.