Great post—I definitely agree with some of your points. I’m very new to LW and haven’t even written an introductory post yet, but I’m very impressed with what I’ve seen overall. I am even flying out to San Francisco tomorrow to discuss joining the newly-renamed CFAR. My background is entirely academic, as I have a PhD in experimental psychology and I’m interested in formalising some of the rationality measures CFAR is looking at. I even had a brief email exchange with Anna Salamon about the usefulness and validity of academic publications.
Here’s my take on your points.
Yes, the time lag is huge. It’s worth noting that in academia papers aren’t used as discussion forums to share ideas rapidly—you read them to see what other labs are doing, and discuss them in journal clubs where graduate students and postdocs can learn about the field and get ideas for their own work. Conference talks and poster presentations are the usual format for discussions between academics from different labs. The academic community as a whole has been very slow to harness the power of the web for the formal discussion of research, although much does take place on science blogs, and now some of the journals are very belatedly starting to host their own discussion forums.
Yes. This is hugely irritating, most academics hate it, and it’s the result of a publishing model that’s massively outdated. Many governments are now taking steps to ensure that publicly-funded research is accessible to all, though we are by no means there yet.
Yes, papers aren’t widely read outside the field they’re published in. But they’re actually not usually supposed to be—the work and the jargon in most fields is so specialised that just being able to read it and understand what people are talking about takes years of training. At that level you need a specialised vocabulary just to be able to ask the questions. Compare it to the jargon used here at LW—it took me a while to get into it, and most of it isn’t completely inaccessible, but it’s still tough for a first-timer to break into this community if they don’t know much about rationalism.
Not sure about this. Philosophers make philosophical arguments in paper form all the time, and they do it primarily to convince other philosophers. If they want to convince people without the formal training, that’s when they’ll usually write a book. (As an experimentalist myself, and given that I’m interested primarily in testing rationality measures, it doesn’t really apply.)
Yes and no to this one. It’s true that writing a paper doesn’t get you mobbed by hordes of screaming fangirls, as I know to my detriment! But again it’s about who you’re trying to convince. You need publications to be taken seriously by other academics, if that’s what you want to do. And if you want to be taken seriously by wider society, there are certainly other ways to do that than by becoming an academic. But as I mentioned before, there’s a vibrant community on science blogs and people interested in science who consider a paper to be the gold standard of scholarship.
Yes, actually you’re supporting the point here I made in my reply to 3. The two tie together quite well in fact.
This is definitely a problem with academia, although I’d consider it more a feature than a bug. Basically the current system is optimised to stop crazy people coming in and messing things up. That is, if you have an idea that’s novel but mainstream you’ll attract funding; if you have an idea that’s way out there you often won’t, people will think you’re a crackpot, etc. etc. GIven the existence of actual crackpots, this is a necessary defence mechanism, although perhaps some knowledge of the base rate of crackpots would be helpful so that academia could make decently Bayesian decisions...
This is true for the moment but as I’ve said I’m interested in joining CFAR (probably not SIAI though, it just doesn’t interest me as much as rationality training) and I’m probably not the only academic who’s at least a bit interested. The more research you can pump out the more either organisation will be seen as a viable place for academics to go and work. I’m sure we can bring things to the table that most non-academics cannot, and can learn things ourselves as well.
Disagree strongly. I suspect you’re right with FAI, which is a topic I’m not very familiar with, but other topics about cognitive biases and heuristics, Bayesian decision theory and so on, have been mainstream in academia for some time now. In fact part of my PhD was based on the premise that the brain treats incoming information in a Bayesian way (see also Tom Griffiths’ work at Berkeley: http://cocosci.berkeley.edu/tom/). So I don’t think it’s correct to say that all of the ideas in this community aren’t academic in origin; it might be better to say that some of them aren’t.
Yes and no. Of course in the abstract you have a conclusion—because the abstract is a tiny mini-paper that you can read to find the answer without having to plough through the whole thing. If you want to check the work, you go and do just that. Papers should be stories—they should flow well and communicate the idea they’re trying to sell. But that idea isn’t written down before the research is done, otherwise there’d be no point in doing the research. It might be written down in the paper before you reach the evidence that supports it, but that’s for narrative reasons. I should note that many papers do start off with a question: “is it A or B? let’s find out!” rather than stating outright whether A or B has won the day. But in reality one of them has already, because the universe is as it is, and the experiments you do to test it only reflect that.
Again this has its good and bad points. A fraction of submissions getting approval goes back to the ‘crackpot filter’ I talked about in my response to 7, which also explains why universities are privileged over other organisations. You don’t want journals filled with any old rubbish, though some journals are obviously better than others (and in your field it very quickly becomes apparent which ones those are). So how do you keep quality up without filtering? You’re right that good papers get rejected and errors slip through—but generally that stuff is caught pretty fast and corrected, and eventually a whole field changes when the errors are brought to its attention. A good example is neuroimaging using fMRI, which a decade ago you could publish anything in really easily but now your stats have to be pretty much watertight. And you’re right that it’s unfair that reviewers get to know who you are but you don’t get to know who they are for the most part… means they can block your grants or your papers if they don’t ’like your ideas or think you’re getting too close to their turf. We are, after all, hierarchical social apes, even those of us who try to be rational.
To summarise then, I generally agree for the most part but there are nuances beyond what you’ve stated in your arguments. I’d agree that papers aren’t great for discussion among the community at the time they’re published, but the advantage is that anyone can go back and find them and other papers and build a narrative from that, much more easily (at the moment) than can be done with a blog. The signal-to-noise ratio is generally higher, although a blog like this with good moderation and smart, curious people ameliorates many problems.
Hello everyone,
Thought it was about time to do one of these since I’ve made a couple of comments!
My name’s Carl. I’ve been interested in science and why people believe the strange things they believe for many years. I was raised Catholic but came to the conclusion around the age of ten that it was all a bit silly really, and as yet I have found no evidence that would cause me to update away from that.
I studied physics as an undergrad and switched to experimental psychology for my PhD, being more interested at that point in how people work than how the universe does. I started to study motor control and after my PhD and a couple of postdocs I know way more about how humans move their arms than any sane person probably should. I’ve worked in behavioural, clinical and computational realms, giving me a wide array of tools to use when analysing problems.
My current postdoc is coming to an end and a couple of months ago I was undergoing somewhat of a crisis. What was I doing, almost 31 and with no plan for my life? I realised that motor control had started to bore me but I had no real idea what to do about it. Stay in science, or abandon it and get a real job? That hurts after almost a decade of high-level research. And then I discovered, on Facebook, a link to HPMOR. And then I read it all, in about a week. And then I found LW, and a job application for curriculum design for a new rationality institute, and I wrote an email, and then flew to San Francisco to participate in the June minicamp...
And now I’m in the midst of writing some fellowship applications to come to Berkeley and study rationality—specifically how the brain is Bayesian in some ways but not in others, and how that can inform the teaching of rationality. (Or something. It’s still in the planning stages!) I’m also volunteering for CFAR at the moment by helping to find useful papers on rationality and cognitive science, though that’s on somewhat of a back burner since these fellowships are due very soon. Next month, in fact.
I’ve started a new blog: it’s called ‘Joy in the Merely Real’, and at the moment I’m exploring a few ideas about the Twelve Virtues of Rationality and what I think about them. You can find it at:
themerelyreal.blogspot.com
Looking forward to doing more with this community in the coming months and years. :)