In general I’m very sympathetic to this point of view, and there are some good examples in your post.
One bad example, in my opinion, is Eliezer’s recent procrastination post vs. the survey of “scientific research on procrastination.” I read the chapter, and it appears to mostly cite studies that involved casual surveys, subjective description, and fuzzy labeling. Although there are many valid scientific endeavors that involve nothing but categorization (it is interesting to know how many species of tree frog there are and what they look and sound like even if we do not make any predictions beyond what is observed), categorization should at least be rigorous enough that we can specify what we expect to see with a modicum of precision.
When a biologist says that frogus neonblueicus has neon blue spots and chirps at 500 Hz, she will give you enough information that you can go to Costa Rica and check for yourself whether you have found one of the rare neonblueicus specimens. Although there will be some controversies around the edges, your identification of any particular frog will not correlate with your political biases or personal problems, and repeated observation of the same frog population by a few different researchers will tend to decrease error.
When a psychologist says that procrastinators can be divided into “relaxed” types and “tense-afraid” types, the “science” being done is not merely descriptive, but also horrifyingly vague. What does it mean for a human to be “tense-afraid” when “procrastinating”? The three paragraphs or so of context on the topic give you enough of an idea of what the researcher is saying to conjure up a mental image, but not nearly enough to carve thing-space at the joints.
In my experience, this is a very serious problem in social and human sciences—there are whole subfields where the authors do not know how little they know, and proceed to wax eloquently about all of the empty concepts they have coined. There are other subfields where the researchers suspect that they might not have done very good research, and they cover their tracks with advanced statistics and jargon. After you dig through a few of these booby-trapped caves of wonder, you start to lose, if not respect for scholarship, at least some of the urge to do the moderately hard work of digesting literature reviews yourself on a regular basis. It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.
Do you think Eliezer’s post is more precise and useful than the controlled experiments published in peer-reviewed journals described in the book I linked to? I find that most writing on psychology is necessarily pretty soft, because the the phenomena it is trying to describe are vastly more complicated than those of the hard sciences.
Now, that link is a must-read. I got through the whole first chapter before I could look away, and I’ll be going back for the rest.
I have nothing against psychology or psychologists or social science in general—AP Psych was my second favorite class in high school, my mom has a master’s degree in it, my bachelor’s degree is in political science, etc. It’s noble, hard work, and we even have a little bit of knowledge to show for it.
As for the “controlled experiments” described in the book you linked to, I’m afraid I missed them, for which I apologize. I only saw descriptive papers. Maybe a page reference when you get a chance? Or just link directly to one or two of the studies or the abstracts?
Oops, you’re right that my link does not mention controlled experiments. A few controlled experiments are instead mentioned in other sections of the book on techniques applicable to a greater variety of behavior change goals.
Unfortunately, the author of Psychological Self-Helpdied last year, and his book has not been updated much in the past decade. Of course, more work on procrastination has been done in recent years, though I’m not sure if it is collected nicely anywhere.
I find that most writing on psychology is necessarily pretty soft, because the the phenomena it is trying to describe are vastly more complicated than those of the hard sciences.
Is there one more step in there? Vastly more complicated → science happens at much higher levels of abstraction → high level abstract science is necessarily pretty soft? Because it seems to me psychology is necessarily soft because it doesn’t want to turn into thirty years of neurobiology before it can talk about human behaviour.
Because it seems to me psychology is necessarily soft because it doesn’t want to turn into thirty years of neurobiology before it can talk about human behaviour.
I hear this sentiment echoed a lot, and I have to admit to either not understanding it or strongly disagreeing with it.
Claiming that psychology has nothing useful to say about human behavior until it can be fully cashed out in neurobiology strikes me as mistaken in many of the same ways that claiming that ballistics has nothing useful to say about missile trajectories until it can be fully cashed out in a relativistic understanding of gravity is.
Yes, our missiles don’t always hit where we want them to, even after thousands of years of work in ballistics. But a deeper understanding of gravity won’t help with that. If we want to improve our practical ability to hit a target, we have to improve our mastery of ballistics at the level of ballistics.
That isn’t quite as true for psychology and neurobiology, granted: the insights afforded by neurobiology often do improve our practical ability to “hit a target.” (Most strikingly in the last few decades, they have allowed us to develop an enormously powerful medical technology for achieving psychological results, which is nothing short of awesome.)
But I think it’s a mistake to conclude from that, that everything about human cognition and behavior can be more usefully described at the level of neurobiology than psychology. There’s a difference between reductionism and greedy reductionism.
If the state of the art in psychology is too soft, too vague, or too contingent, then the goal to strive for is to make it more rigorous, more specific, or more reliable… not to give it up altogether and work exclusively in neurobiology instead.
Claiming that psychology has nothing useful to say about human behavior until it can be fully cashed out in neurobiology strikes me as mistaken
Ha, no, I’m on your side. Psychology can say useful things precisely because it isn’t cashed out in neurobiology. The point I was making was that in order to have simple rules for brains, in all their hundred-billion-neuron complexity, you need to have softer edges on your predictions.
I don’t mean soft in any derogatory way. The concept I was aiming for was something like Eliezer’s “well, you could simulate an aeroplane prototype from the quark-level up, but that’s inefficient. The field of aerodynamics has good approximations for that macro-scale behaviour,”: even if you did drop psychology for neurobiology, describing human behaviour from the neuron-level up is inefficient. Psychology is soft in that it uses approximations of human behaviour, in order to be useful on human timescales. This is a good thing, made no worse by the fact that it necessitates some level of ’soft’ness.
(I think the concern some people have with psychology is that they perceive it as too soft. Availability bias has them drawing generalisations from the describes-everything Freudian analysis, and so forth.)
I stand by my response in and of itself, but I sheepishly admit that it’s not actually a response to you at all. Rereading your comment, I conclude that I was overtrained on the kind of objections I responded to, which you didn’t actually make… sorry about that.
Doesn’t bother me in the slightest. In fact, I almost included another parenthetical:
(Hard scientists probably do think hard is good and soft is bad, but that’s because they’re hard scientists. Soft scientists are probably sensitive to the negative connotations the hard scientists attach to these terms, because there is something of a rivalry between hard and soft science.)
I guess you’ve studied some kind of soft science at a college or university?
(I feel like I have overused the terms, though. I make sound as if there is a strict divide, when in my mind it’s an evenly distributed spectrum.)
I think it is more: Complication allows the researchers’ biases to slip in more easily, since among other things any sort of cross-check is nearly impossible, which leads to softer results, especially when being evaluated by someone with different biases.
The book you linked to is mostly irrelevant to the problem Eliezer was addressing. The author writes, “Both types of procrastinators dislike the chores they are avoiding.” Eliezer’s hypothesis is a contribution even if (like me) you don’t think it true. Eliezer recognized that ordinary hyperbolic discounting can’t explain procrastination such as he experiences, where he decidedly does not dislike the activities, which can’t be described as “chores.” His clever solution is to apply hyperbolic-discounting considerations to mental acts.
I don’t think it’s accurate to say Eliezer posted in ignorance of the literature on procrastination. Everything the book you linked to mentions is well-known, truistic by now, except the distinction between relaxed and tense procrastinators—a dispensable classification.
Hyperbolic discounting is pretty much clearly the correct overarching framework for the kind of procrastination the author of the linked book discusses—but you don’t learn that from the linked book (unless I missed it).
It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.
However, enough rationality training will have alarm bells ringing when reading soft textbooks and studies. That in itself—“this field is overpopulated with concepts and undermeasured”—is marginally more useful than knowing nothing about the field.
I was a philosophy student for my brief attempt at tertiary education—I know what you mean. Our lecturer would describe the text as ‘dense’ - more aptly, I thought, the author is dense.
An anecdote from that class: after a lecture on Wittgenstein, a student asked the lecturer if the rest of the semester’s lectures were to be canceled.
There is an obvious one, actually—a frequent (perhaps inaccurate) interpretation of the last parts of the Tractatus is as a denial of the possibility of any real philosophy (including Wittgenstein’s).
Since one would naturally cover the Tractatus before The Philosophical Investigations or other works, a rather juvenile response would be exactly that anecdote.
Yep. The lecture presented the view that Wittgenstein had explained away most of philosophy—in his own words, that he had resolved all philosophical problems.
Oh, Hegel. I remember a lecture where the professor read from Hegel’s Wissenschaft der Logik like it was a holy scripture. When he was finished, he looked up and said: “With this, everything is said”. I didn’t understand anything, it was a jungle of words like being and not-being and becoming and how one thing becomes the other. I said that I didn’t understand anything, and what did the lecturer reply with a smile? “It’s good you don’t understand it!” I seriously had the intense urge to shout at him, but instead I just didn’t show up anymore.
A perhaps equally juvenile concern of mine, is whether Wittgenstein himself failed to stand on the shoulders of giants (at least in the Tractatus), by essentially starting from scratch with his own propositions, drawing logical conclusions from them rather than using or at least referring to previous work.
In general I’m very sympathetic to this point of view, and there are some good examples in your post.
One bad example, in my opinion, is Eliezer’s recent procrastination post vs. the survey of “scientific research on procrastination.” I read the chapter, and it appears to mostly cite studies that involved casual surveys, subjective description, and fuzzy labeling. Although there are many valid scientific endeavors that involve nothing but categorization (it is interesting to know how many species of tree frog there are and what they look and sound like even if we do not make any predictions beyond what is observed), categorization should at least be rigorous enough that we can specify what we expect to see with a modicum of precision.
When a biologist says that frogus neonblueicus has neon blue spots and chirps at 500 Hz, she will give you enough information that you can go to Costa Rica and check for yourself whether you have found one of the rare neonblueicus specimens. Although there will be some controversies around the edges, your identification of any particular frog will not correlate with your political biases or personal problems, and repeated observation of the same frog population by a few different researchers will tend to decrease error.
When a psychologist says that procrastinators can be divided into “relaxed” types and “tense-afraid” types, the “science” being done is not merely descriptive, but also horrifyingly vague. What does it mean for a human to be “tense-afraid” when “procrastinating”? The three paragraphs or so of context on the topic give you enough of an idea of what the researcher is saying to conjure up a mental image, but not nearly enough to carve thing-space at the joints.
In my experience, this is a very serious problem in social and human sciences—there are whole subfields where the authors do not know how little they know, and proceed to wax eloquently about all of the empty concepts they have coined. There are other subfields where the researchers suspect that they might not have done very good research, and they cover their tracks with advanced statistics and jargon. After you dig through a few of these booby-trapped caves of wonder, you start to lose, if not respect for scholarship, at least some of the urge to do the moderately hard work of digesting literature reviews yourself on a regular basis. It is dangerous to assume that casually studying the leading textbook in a soft field will usually make you smarter.
I very much agree with your final sentence.
Do you think Eliezer’s post is more precise and useful than the controlled experiments published in peer-reviewed journals described in the book I linked to? I find that most writing on psychology is necessarily pretty soft, because the the phenomena it is trying to describe are vastly more complicated than those of the hard sciences.
Now, that link is a must-read. I got through the whole first chapter before I could look away, and I’ll be going back for the rest.
I have nothing against psychology or psychologists or social science in general—AP Psych was my second favorite class in high school, my mom has a master’s degree in it, my bachelor’s degree is in political science, etc. It’s noble, hard work, and we even have a little bit of knowledge to show for it.
As for the “controlled experiments” described in the book you linked to, I’m afraid I missed them, for which I apologize. I only saw descriptive papers. Maybe a page reference when you get a chance? Or just link directly to one or two of the studies or the abstracts?
Oops, you’re right that my link does not mention controlled experiments. A few controlled experiments are instead mentioned in other sections of the book on techniques applicable to a greater variety of behavior change goals.
Unfortunately, the author of Psychological Self-Help died last year, and his book has not been updated much in the past decade. Of course, more work on procrastination has been done in recent years, though I’m not sure if it is collected nicely anywhere.
Is there one more step in there? Vastly more complicated → science happens at much higher levels of abstraction → high level abstract science is necessarily pretty soft? Because it seems to me psychology is necessarily soft because it doesn’t want to turn into thirty years of neurobiology before it can talk about human behaviour.
I hear this sentiment echoed a lot, and I have to admit to either not understanding it or strongly disagreeing with it.
Claiming that psychology has nothing useful to say about human behavior until it can be fully cashed out in neurobiology strikes me as mistaken in many of the same ways that claiming that ballistics has nothing useful to say about missile trajectories until it can be fully cashed out in a relativistic understanding of gravity is.
Yes, our missiles don’t always hit where we want them to, even after thousands of years of work in ballistics. But a deeper understanding of gravity won’t help with that. If we want to improve our practical ability to hit a target, we have to improve our mastery of ballistics at the level of ballistics.
That isn’t quite as true for psychology and neurobiology, granted: the insights afforded by neurobiology often do improve our practical ability to “hit a target.” (Most strikingly in the last few decades, they have allowed us to develop an enormously powerful medical technology for achieving psychological results, which is nothing short of awesome.)
But I think it’s a mistake to conclude from that, that everything about human cognition and behavior can be more usefully described at the level of neurobiology than psychology. There’s a difference between reductionism and greedy reductionism.
If the state of the art in psychology is too soft, too vague, or too contingent, then the goal to strive for is to make it more rigorous, more specific, or more reliable… not to give it up altogether and work exclusively in neurobiology instead.
Ha, no, I’m on your side. Psychology can say useful things precisely because it isn’t cashed out in neurobiology. The point I was making was that in order to have simple rules for brains, in all their hundred-billion-neuron complexity, you need to have softer edges on your predictions.
I don’t mean soft in any derogatory way. The concept I was aiming for was something like Eliezer’s “well, you could simulate an aeroplane prototype from the quark-level up, but that’s inefficient. The field of aerodynamics has good approximations for that macro-scale behaviour,”: even if you did drop psychology for neurobiology, describing human behaviour from the neuron-level up is inefficient. Psychology is soft in that it uses approximations of human behaviour, in order to be useful on human timescales. This is a good thing, made no worse by the fact that it necessitates some level of ’soft’ness.
(I think the concern some people have with psychology is that they perceive it as too soft. Availability bias has them drawing generalisations from the describes-everything Freudian analysis, and so forth.)
(nods) Fair enough, and agreed throughout.
I stand by my response in and of itself, but I sheepishly admit that it’s not actually a response to you at all. Rereading your comment, I conclude that I was overtrained on the kind of objections I responded to, which you didn’t actually make… sorry about that.
Doesn’t bother me in the slightest. In fact, I almost included another parenthetical:
(Hard scientists probably do think hard is good and soft is bad, but that’s because they’re hard scientists. Soft scientists are probably sensitive to the negative connotations the hard scientists attach to these terms, because there is something of a rivalry between hard and soft science.)
I guess you’ve studied some kind of soft science at a college or university?
(I feel like I have overused the terms, though. I make sound as if there is a strict divide, when in my mind it’s an evenly distributed spectrum.)
I think it is more: Complication allows the researchers’ biases to slip in more easily, since among other things any sort of cross-check is nearly impossible, which leads to softer results, especially when being evaluated by someone with different biases.
The book you linked to is mostly irrelevant to the problem Eliezer was addressing. The author writes, “Both types of procrastinators dislike the chores they are avoiding.” Eliezer’s hypothesis is a contribution even if (like me) you don’t think it true. Eliezer recognized that ordinary hyperbolic discounting can’t explain procrastination such as he experiences, where he decidedly does not dislike the activities, which can’t be described as “chores.” His clever solution is to apply hyperbolic-discounting considerations to mental acts.
I don’t think it’s accurate to say Eliezer posted in ignorance of the literature on procrastination. Everything the book you linked to mentions is well-known, truistic by now, except the distinction between relaxed and tense procrastinators—a dispensable classification.
Hyperbolic discounting is pretty much clearly the correct overarching framework for the kind of procrastination the author of the linked book discusses—but you don’t learn that from the linked book (unless I missed it).
However, enough rationality training will have alarm bells ringing when reading soft textbooks and studies. That in itself—“this field is overpopulated with concepts and undermeasured”—is marginally more useful than knowing nothing about the field.
If you haven’t already, you should try reading postmodern philosophy. An uninterrupted wall of alarm bells. :)
I was a philosophy student for my brief attempt at tertiary education—I know what you mean. Our lecturer would describe the text as ‘dense’ - more aptly, I thought, the author is dense.
An anecdote from that class: after a lecture on Wittgenstein, a student asked the lecturer if the rest of the semester’s lectures were to be canceled.
I cannot think of a single obvious interpretation for why this occurred, but I can think of a few possible ones. Could you please clarify?
There is an obvious one, actually—a frequent (perhaps inaccurate) interpretation of the last parts of the Tractatus is as a denial of the possibility of any real philosophy (including Wittgenstein’s).
Since one would naturally cover the Tractatus before The Philosophical Investigations or other works, a rather juvenile response would be exactly that anecdote.
Yep. The lecture presented the view that Wittgenstein had explained away most of philosophy—in his own words, that he had resolved all philosophical problems.
How silly of Wittgenstein! Didn’t he know that Hegel had already completed philosophy?
Oh, Hegel. I remember a lecture where the professor read from Hegel’s Wissenschaft der Logik like it was a holy scripture. When he was finished, he looked up and said: “With this, everything is said”. I didn’t understand anything, it was a jungle of words like being and not-being and becoming and how one thing becomes the other. I said that I didn’t understand anything, and what did the lecturer reply with a smile? “It’s good you don’t understand it!” I seriously had the intense urge to shout at him, but instead I just didn’t show up anymore.
A perhaps equally juvenile concern of mine, is whether Wittgenstein himself failed to stand on the shoulders of giants (at least in the Tractatus), by essentially starting from scratch with his own propositions, drawing logical conclusions from them rather than using or at least referring to previous work.
Perhaps my choosing a recent Eliezer article as one example of an underuse of schholarship is an instance of “people trying to show off how willing they are to disagree with” Eliezer Yudkowsky!