Only write up the obvious conclusions from your case-studies. I had plenty of subtle theories to explain Jordan’s policies, but just a few which strong evidence. I asked myself “if a young man from Mafraq reproduced my methodology, would he arrive at the same conclusions”, and left out ideas which failed the test.
This feels like the traditional academic approach, but it results in at least a couple of weird effects based on patterns we see in academic publishing:
a large amount of unpublished ideas that are known to insiders because they are shared only informally but still influence the results published in the field in a way that is opaque to outsiders and beyond comment/consideration
faking or exaggerating data/results in order to reach publication standards of evidence
This suggests attempts to reform academic publishing norms might be relevant here.
a large amount of unpublished ideas that are known to insiders because they are shared only informally but still influence the results published in the field in a way that is opaque to outsiders and beyond comment/consideration
That is a great point,. If I were describing my results to another expert who understood bayesian reasoning, I would speak differently. Perhaps I will do a writeup in that framework.
faking or exaggerating data/results in order to reach publication standards of evidence
so fucking true. Or dropping disconfirming evidence, which is easy to do. I had peer reviewers ask me to do this. If I find time, I will post an anonymized quote.
This feels like the traditional academic approach, but it results in at least a couple of weird effects based on patterns we see in academic publishing:
a large amount of unpublished ideas that are known to insiders because they are shared only informally but still influence the results published in the field in a way that is opaque to outsiders and beyond comment/consideration
faking or exaggerating data/results in order to reach publication standards of evidence
This suggests attempts to reform academic publishing norms might be relevant here.
That is a great point,. If I were describing my results to another expert who understood bayesian reasoning, I would speak differently. Perhaps I will do a writeup in that framework.
so fucking true. Or dropping disconfirming evidence, which is easy to do. I had peer reviewers ask me to do this. If I find time, I will post an anonymized quote.