As as as I know there no evidence for the thesis that thinking analytically always provides an improvement.
Not only did I not claim thinking analytically in the manner I’m describing always provides an improvement, I noted multiple exceptions when it doesn’t improve it.
A lot of expert thinking isn’t analytic.
If you want specific examples of what I’m referring to in regards to scientists screwing up, read Andrew Gelman’s or James Coyne’s blogs.
Do you think you engaged into that type of thinking while writing this post?
I noted multiple exceptions when it doesn’t improve it.
It sounded to me like you were criticizing doctors for not being scientific. Is your thesis that it’s alright that doctors aren’t?
If you want specific examples of what I’m referring to in regards to scientists screwing up, read Andrew Gelman’s or James Coyne’s blogs.
So on the one hand you advocate scientific thinking and then you come and provide anecdotal evidence? Decision science is a field.
This is just vague.
Why is the question of whether a particular form of reasoning follows the standards of “Thinking like a Scientist” vague?
If you would have clear standards of what “Thinking like a Scientist” means, I think you should be able to answer the question.
If you don’t have clear standards, then you are right it’s vague. That means there a problem. Being to vague to be wrong is bad. The paradigm of science would say that removing vagueness from your theory would improve it.
“What do you actually mean when you say ‘Thinking like a Scientist’” is to me the better response than ’Yeah science”.
I do not claim that in-depth scientific analysis is always necessary. I claim that heuristics are often the rational approach, and gave examples in which I stated they were rational. I do not claim that all or even most scientists screw up their analysis. I do not claim how frequently it occurs. I do claim that I see it often enough that it worries me. I do claim that among non-scientists, the problem is much more common. I do not claim that personal experiences are useless. I claim that statistical analysis is superior and that people will often not note this in their personal assessment. I claim that I see these problems often enough that I can determine heuristically that it is likely to be extremely common.
I do claim that among non-scientists, the problem is much more common.
Non-scientists don’t often engage in writing scientific papers. In what instances do you believe they screw things up and should engage in in-depth scientific analysis?
Reasoning from a statistical outlier is probably the most common error I see. Most news stories do this.
Mistakes on LessWrong tend to be more varied. Likely due to LessWrong’s focus on Cognitive Psychology, I see a lot of people repeating the mistakes of Psychoanalysis; coming up with elaborate theories about mental processes with no reasonable means of verification of their ideas.
They do this because it’s good storytelling and they want to sell papers. A have fairly low confidence that teaching the authors statistics helps in any way.
If you think it helps can you explain why you think so?
coming up with elaborate theories about mental processes with no reasonable means of verification of their ideas.
Why do you think those posts need “reasonable means of verification of their ideas” while you haven’t provided one for the post you written and think it’s okay based on heuristics? Aren’t those people not also simply using heuristics instead of structured scientific thinking?
Not the authors; the readers; though I don’t think the authors are generally aware of the problem either.
Verifiability is not heuristics. I’m combining in the term verification the two scientific concepts of direct observation and falsification. By elaborate theories, I’m referring to occam’s razor.
My post isn’t a theory post. It contains a few ideas and a lot of observations, but the assumptions are pretty straightforward and they’re related to the central concept of scientific analysis, but not dependent on one another. The general statement about schools is not dependent on the specific statement about math, nor is the argument about math dependent on the argument about whether rationality is sufficient. And I try to be exact with my phrasing to specify my uncertainty where it exists.
Not only did I not claim thinking analytically in the manner I’m describing always provides an improvement, I noted multiple exceptions when it doesn’t improve it.
If you want specific examples of what I’m referring to in regards to scientists screwing up, read Andrew Gelman’s or James Coyne’s blogs.
This is just vague.
It sounded to me like you were criticizing doctors for not being scientific. Is your thesis that it’s alright that doctors aren’t?
So on the one hand you advocate scientific thinking and then you come and provide anecdotal evidence? Decision science is a field.
Why is the question of whether a particular form of reasoning follows the standards of “Thinking like a Scientist” vague? If you would have clear standards of what “Thinking like a Scientist” means, I think you should be able to answer the question.
If you don’t have clear standards, then you are right it’s vague. That means there a problem. Being to vague to be wrong is bad. The paradigm of science would say that removing vagueness from your theory would improve it.
“What do you actually mean when you say ‘Thinking like a Scientist’” is to me the better response than ’Yeah science”.
I do not claim that in-depth scientific analysis is always necessary. I claim that heuristics are often the rational approach, and gave examples in which I stated they were rational. I do not claim that all or even most scientists screw up their analysis. I do not claim how frequently it occurs. I do claim that I see it often enough that it worries me. I do claim that among non-scientists, the problem is much more common. I do not claim that personal experiences are useless. I claim that statistical analysis is superior and that people will often not note this in their personal assessment. I claim that I see these problems often enough that I can determine heuristically that it is likely to be extremely common.
Non-scientists don’t often engage in writing scientific papers. In what instances do you believe they screw things up and should engage in in-depth scientific analysis?
Reasoning from a statistical outlier is probably the most common error I see. Most news stories do this.
Mistakes on LessWrong tend to be more varied. Likely due to LessWrong’s focus on Cognitive Psychology, I see a lot of people repeating the mistakes of Psychoanalysis; coming up with elaborate theories about mental processes with no reasonable means of verification of their ideas.
They do this because it’s good storytelling and they want to sell papers. A have fairly low confidence that teaching the authors statistics helps in any way. If you think it helps can you explain why you think so?
Why do you think those posts need “reasonable means of verification of their ideas” while you haven’t provided one for the post you written and think it’s okay based on heuristics? Aren’t those people not also simply using heuristics instead of structured scientific thinking?
Not the authors; the readers; though I don’t think the authors are generally aware of the problem either.
Verifiability is not heuristics. I’m combining in the term verification the two scientific concepts of direct observation and falsification. By elaborate theories, I’m referring to occam’s razor.
My post isn’t a theory post. It contains a few ideas and a lot of observations, but the assumptions are pretty straightforward and they’re related to the central concept of scientific analysis, but not dependent on one another. The general statement about schools is not dependent on the specific statement about math, nor is the argument about math dependent on the argument about whether rationality is sufficient. And I try to be exact with my phrasing to specify my uncertainty where it exists.