A register of proposals does seem like it would help to keep scientists honest, a step towards Feynman’s “utter honesty.” I would hesitate to say that journals should accept papers based solely on that register, though. Sometimes, the proposal might not end up as a wholly accurate description of the actual experiment, for a variety of reasons. I think that making both available would be a good way to judge how well the result actually applies to what is claimed by the scientists who published it. They could offer explanations for any differences between the proposal and the actual, and peer reviewers could give more thorough critiques with this extra information.
What’s the reason to not demand that all experiments be videoed in their entirety?
You seem to be trying to accommodate the way scientists and journals already operate:
Sometimes, the proposal might not end up as a wholly accurate description of the actual experiment, for a variety of reasons.
It might not be bad to accommodate them, but the primary and central purpose of science is to know – to produce shared knowledge of the world.
I think an ideal journal would allow scientists to change their registered proposal – possibly. That would also be recorded in the journal’s register, if they accept the changes.
Maybe I’m in a bad mood, but it’s especially galling how terrible all of this still, e.g. NOT sharing all scientific results with the public for publicly funded research.
Why can’t all of this be done in the open, on the researcher’s blog? They register a proposal by publishing a post describing it, in as much detail as is feasible, e.g. including code they’re registering to use on the data they collect. They record video of the entire experiment (where feasible); they publish that to YouTube. They publish all of their data. They perform their analysis – the exact one described in their registration post – and then publish a blog post, or a whole series of posts, about their analysis.
If the researchers want to change a registered, but un-performed, experiment, they publish a post describing their changes, in comparable detail as originally.
Blog posts don’t need to be open for anyone to comment on. Researchers could explicitly invite other individuals or ‘anyone with X degree in Y from an accredited institution recognized by professional association Z’.
The relevant people could comment on the registered proposal, on registered changes, on the documentation of the performance of the experiment itself, and on interpretation of the registered analysis.
A register of proposals does seem like it would help to keep scientists honest, a step towards Feynman’s “utter honesty.” I would hesitate to say that journals should accept papers based solely on that register, though. Sometimes, the proposal might not end up as a wholly accurate description of the actual experiment, for a variety of reasons. I think that making both available would be a good way to judge how well the result actually applies to what is claimed by the scientists who published it. They could offer explanations for any differences between the proposal and the actual, and peer reviewers could give more thorough critiques with this extra information.
What’s the reason to not demand that all experiments be videoed in their entirety?
You seem to be trying to accommodate the way scientists and journals already operate:
It might not be bad to accommodate them, but the primary and central purpose of science is to know – to produce shared knowledge of the world.
I think an ideal journal would allow scientists to change their registered proposal – possibly. That would also be recorded in the journal’s register, if they accept the changes.
Maybe I’m in a bad mood, but it’s especially galling how terrible all of this still, e.g. NOT sharing all scientific results with the public for publicly funded research.
Why can’t all of this be done in the open, on the researcher’s blog? They register a proposal by publishing a post describing it, in as much detail as is feasible, e.g. including code they’re registering to use on the data they collect. They record video of the entire experiment (where feasible); they publish that to YouTube. They publish all of their data. They perform their analysis – the exact one described in their registration post – and then publish a blog post, or a whole series of posts, about their analysis.
If the researchers want to change a registered, but un-performed, experiment, they publish a post describing their changes, in comparable detail as originally.
Blog posts don’t need to be open for anyone to comment on. Researchers could explicitly invite other individuals or ‘anyone with X degree in Y from an accredited institution recognized by professional association Z’.
The relevant people could comment on the registered proposal, on registered changes, on the documentation of the performance of the experiment itself, and on interpretation of the registered analysis.
Why do we need journals? Why do we want journals?