PZ’s his own special brand of abrasive and dismissive, but I went and read most of the paper, and while he’s not exactly rigorous with explaining his criticisms, I think they’re based in good ones.
While the design of the JoC website shouldn’t affect assessment of the article, the fact that a paper on such a potentially high-impact subject isn’t in a mainstream journal at all does and should send up some red flags that there might be issues with the paper that would keep it from getting past peer review.
My biggest issue with the paper is that the study isn’t controlled. They took appropriate steps to prevent contamination of their samples, but they don’t have any reasonable negative control set up that would give them some perspective on their comparison to the living bacteria.
Their “conclusions” are suggestive, rather than conclusive—it all depends on holding up these meteorites to pictures of actual bacteria and saying “Look! They look alike! And there’s some enriched carbon and stuff in these fossils!” Which could certainly be interesting, but for the paper to pass muster with mainstream science, they would need to offer a convincing test that would disprove their hypothesis were it to come out a certain way. (Hey, that sounds familiar!) As it stands, they can’t. They can only say that their observations look interesting.
Given this, the whole thing reads like they went out looking for whatever evidence they could fit to their prior hope of finding extraterrestrial life, which doesn’t immediately disprove their findings, but it certainly holds them back from credibility.
Hanson has a post on this. It seems to me that the chemical signature matching ancient life much better than recent life is really quite strong evidence.
I’d bet these commentaries will mostly say this is interesting but doubts remain, that this evidence is too ordinary to support its “extraordinary” conclusion, and yet they’ll refuse to bet on the subject. How sad is that?
I’d really like to see Hanson put his money where his mouth is as in a clear, unambiguous and publicly available way. A decent figure (>=$1,000) at odds he considers favorable after considering this evidence and all his priors. That would be powerful advocacy of his most notable economics related there (prediction via markets).
This does not seem to be something that is especially easy to bet on practically. That is it would tie up capital basically until it was too late for Hanson to use it (he isn’t especially young). A way to make long term bets like this in which the money is also invested in an index fund in the interim and pays dividends would be a valuable public service!
I’d love to see some series of posts where someone applies LWism to such stories. LW is too much about refining rationality when the biggest problem is that people don’t know how to apply it to real life problems. But since Yvain has basically bashed that idea I lost all hope that it will ever happen. He actually got 26 upvotes for it and I don’t even know of a single post that shows how to apply what you can learn on LW to real life problems.
Looking back on that comment I agree it came across as too harsh.
I meant only that this site’s comparative advantage is not in teaching object-level information about how to succeed socially, pick up girls/boys, program computers, or lose weight—even if the post with the information is called “Losing Weight The Rationalist Way”.
I totally support more posts on the interaction between rationality and real-life problems, like if there is a specific cognitive bias that’s preventing people from losing weight, or if the problem of weight loss illuminates some wider issue about how people think and act. Just not “Hey guys, I just read about how if you only eat carrots for a week you can lose thirty pounds! This sounds really rational and high utility! Bayes!”
(I feel the same way about discussion of politics. I realize different people’s intuition about where to draw the line on weight-loss and politics posts are in different places, and I know I’ve gotten flak for crossing that line a few times, and I’m willing to live and let live on the issue up until it becomes really blatant.)
I would welcome an analysis of this claim, but I think it would be most suited to Less Wrong if the analysis focused on the process of analyzing scientific papers, rather than just being about the biology of cyanobacteria. I’d probably still read the cyanobacteria one, I just wouldn’t see why it’s on LW.
less about the application of rationality to various different everyday life skills (which usually end up being bad self-help)
Which is an objection to all of the posts about akrasia, social skills, and so forth, not to applying rationality to figure out whether particular scientific claims are likely to be true.
I agree the comment seems somewhat off-topic, but I’m glad it was posted here anyway only because I’m new and was considering posting things similar to what you say is discouraged. But I would like to ask, is the consensus that the topic itself is discouraged, or that if you are to post on that topic, the bar should be set much higher? If anyone has advice on what you guys want to see, or not see, here, it would be appreciated.
There have been a bunch of posts on self-help related topics (akrasia, social skills, etc.), and some of them have been very well received, so if that’s the topic that you’re referring to then it could fit in fine. Yvain was just giving his own views, not setting official LW policy (and he’s clarified his views with another comment here).
If you want feedback on whether people would be interested in your post, you could give a two sentence summary of it, either here or in the open thread.
There’s no real limit on what you can get away with if you can clear a high enough bar. For example, here’s an article about discovering the sunken city of Atlantis.
There’s a lot of reason to be skeptical of these claims. See these remarks by PZ Myers.
Here are the reasons to be skeptical that I picked up from that blog post:
The website of the Journal of Cosmology is ugly
The figures in the paper are “annoying”
Perhaps the claimed bacteria aren’t bacteria at all, but just squiggles.
The photos of the found bacteria aren’t at the same magnification as photos of real bacteria
It seems like the bacteria are too well-preserved for having traveled the solar system for such a long time.
Haha, maybe next they’ll find bigfoot footprints on a meteor.
PZ’s his own special brand of abrasive and dismissive, but I went and read most of the paper, and while he’s not exactly rigorous with explaining his criticisms, I think they’re based in good ones.
While the design of the JoC website shouldn’t affect assessment of the article, the fact that a paper on such a potentially high-impact subject isn’t in a mainstream journal at all does and should send up some red flags that there might be issues with the paper that would keep it from getting past peer review.
My biggest issue with the paper is that the study isn’t controlled. They took appropriate steps to prevent contamination of their samples, but they don’t have any reasonable negative control set up that would give them some perspective on their comparison to the living bacteria.
Their “conclusions” are suggestive, rather than conclusive—it all depends on holding up these meteorites to pictures of actual bacteria and saying “Look! They look alike! And there’s some enriched carbon and stuff in these fossils!” Which could certainly be interesting, but for the paper to pass muster with mainstream science, they would need to offer a convincing test that would disprove their hypothesis were it to come out a certain way. (Hey, that sounds familiar!) As it stands, they can’t. They can only say that their observations look interesting.
Given this, the whole thing reads like they went out looking for whatever evidence they could fit to their prior hope of finding extraterrestrial life, which doesn’t immediately disprove their findings, but it certainly holds them back from credibility.
Typical PZ Myers.
Hanson has a post on this. It seems to me that the chemical signature matching ancient life much better than recent life is really quite strong evidence.
I like this quote from Hanson:
I’d really like to see Hanson put his money where his mouth is as in a clear, unambiguous and publicly available way. A decent figure (>=$1,000) at odds he considers favorable after considering this evidence and all his priors. That would be powerful advocacy of his most notable economics related there (prediction via markets).
This does not seem to be something that is especially easy to bet on practically. That is it would tie up capital basically until it was too late for Hanson to use it (he isn’t especially young). A way to make long term bets like this in which the money is also invested in an index fund in the interim and pays dividends would be a valuable public service!
Yeah, I’m pretty damn skeptical. See here. But, we can wait and see what comes of it.
There’s more at Bad Astronomy.
I’d love to see some series of posts where someone applies LWism to such stories. LW is too much about refining rationality when the biggest problem is that people don’t know how to apply it to real life problems. But since Yvain has basically bashed that idea I lost all hope that it will ever happen. He actually got 26 upvotes for it and I don’t even know of a single post that shows how to apply what you can learn on LW to real life problems.
Looking back on that comment I agree it came across as too harsh.
I meant only that this site’s comparative advantage is not in teaching object-level information about how to succeed socially, pick up girls/boys, program computers, or lose weight—even if the post with the information is called “Losing Weight The Rationalist Way”.
I totally support more posts on the interaction between rationality and real-life problems, like if there is a specific cognitive bias that’s preventing people from losing weight, or if the problem of weight loss illuminates some wider issue about how people think and act. Just not “Hey guys, I just read about how if you only eat carrots for a week you can lose thirty pounds! This sounds really rational and high utility! Bayes!”
(I feel the same way about discussion of politics. I realize different people’s intuition about where to draw the line on weight-loss and politics posts are in different places, and I know I’ve gotten flak for crossing that line a few times, and I’m willing to live and let live on the issue up until it becomes really blatant.)
I would welcome an analysis of this claim, but I think it would be most suited to Less Wrong if the analysis focused on the process of analyzing scientific papers, rather than just being about the biology of cyanobacteria. I’d probably still read the cyanobacteria one, I just wouldn’t see why it’s on LW.
That’s not what Yvain said. He called for:
Which is an objection to all of the posts about akrasia, social skills, and so forth, not to applying rationality to figure out whether particular scientific claims are likely to be true.
I agree the comment seems somewhat off-topic, but I’m glad it was posted here anyway only because I’m new and was considering posting things similar to what you say is discouraged. But I would like to ask, is the consensus that the topic itself is discouraged, or that if you are to post on that topic, the bar should be set much higher? If anyone has advice on what you guys want to see, or not see, here, it would be appreciated.
There have been a bunch of posts on self-help related topics (akrasia, social skills, etc.), and some of them have been very well received, so if that’s the topic that you’re referring to then it could fit in fine. Yvain was just giving his own views, not setting official LW policy (and he’s clarified his views with another comment here).
If you want feedback on whether people would be interested in your post, you could give a two sentence summary of it, either here or in the open thread.
There’s no real limit on what you can get away with if you can clear a high enough bar. For example, here’s an article about discovering the sunken city of Atlantis.
As the author of one such post I have to protest this exaggeration, even while I agree that it would be nice to see more.
I’ve actually read it forwarded that life originates in outer space, because the conditions necessary to form it are not found on planets.
I’m not four billion years old, so I can’t verify it.