Do-it-yourself-science Wiki
While I was busy procrastinating, I produced http://38020.vs.webtropia.com/sciencewiki/index.php/Helium_balloon (the Wiki-Extension for the plot, not the data, that was lying around anyway). This could (if enough people are using it) become quite a useful collection of evidence of various simple sciency questions, and also ultimately motivate some more people to actually do experiments themselves.
Before going further with this (in particular wrt. telling people about it) however, I have a few questions:
Is anybody aware of a similar effort to collect data from hobby scientists in formalized, yet wiki enabled form? I googled, but found nothing, in that case I’d rather not start a new project.
I would like to calculate a metric evaluating which models are “better”, i.e. explain the data best, yet are not overfitted. Can anybody recommend a paper or book about this problem? In particular, I need a metric which can handle errors in all variables (not just the dependent ones), and would rather not like to assume a normal distribution globally.
General feedback is obviously also welcome. If anybody has data which needs something different from a scatterplot, just throw it in, I’ll see that a decent plot gets implemented.
Data without an explanation of how it collected is mostly useless. Most of the text that goes into papers is actually useful- you could explain what the heck a carrying capacity of −1 grams is.
Systematic error can be pretty massive, and so if a hobbyist produces interesting results there’s no guarantee they’re seeing something real. Combine that with limited oversight or experience in experimental design, and I’m pessimistic about the quality of the results you’ll get.
In my experience, do-it-yourself engineering has a better record than do-it-yourself science; science is really hard to get right and less satisfying (if your primary desire is encouraging the scientific/engineering viewpoint).