Thanks for your reply. Given my own time constraints I’ll decline your kind offer to discuss this further (I would be interested in reading some future synthesis). As consolation, I’d happily take you up on the modified bet. Something like:
Within the next 24 months CFAR will not produce results of sufficient quality for academic publication (as judged by someone like Christiano or Karnofsky) that demonstrate benefit on a pre-specified objective outcome measure
I guess ‘demonstrate benefit’ could be stipulated as ‘p<0.05 on some appropriate statistical test’ (the pre-specification should get rid of the p-hacking worries). ‘Objective’ may remain a bit fuzzy: the rider is meant to rule out self-report stuff like “Participants really enjoyed the session/thought it helped them”. I’d be happy to take things like “Participants got richer than controls”, “CFAR alums did better on these previously used metrics of decision making”, or whatever else.
Happy to discuss further to arrive at agreeable stipulations—or, if you prefer, we can just leave them to the judges discretion.
(4 to 1): Conditional on a CFAR study getting past peer review, it will not show significantly positive effects on any objective, pre-specified outcome measure.
I don’t know CFAR’s current plans well enough to judge whether they will synthesize the relevant evidence. I am only betting that if they do, the result will be positive. I am still on the fence of taking a 4-1 bet on this, but the vast majority of my uncertainty here comes from what CFAR is planning to do, not what the result would be. I would probably take a 5-1 bet on the statement as you proposed it.
Sorry for misreading your original remark. Happy to offer the bet in conditional, i.e.:
Conditional on CFAR producing results of sufficient quality for academic publication (as judged by someone like Christiano or Karnofsky) these will fail to demonstrate benefit on a pre-specified objective outcome measure
Thanks for your reply. Given my own time constraints I’ll decline your kind offer to discuss this further (I would be interested in reading some future synthesis). As consolation, I’d happily take you up on the modified bet. Something like:
Within the next 24 months CFAR will not produce results of sufficient quality for academic publication (as judged by someone like Christiano or Karnofsky) that demonstrate benefit on a pre-specified objective outcome measure
I guess ‘demonstrate benefit’ could be stipulated as ‘p<0.05 on some appropriate statistical test’ (the pre-specification should get rid of the p-hacking worries). ‘Objective’ may remain a bit fuzzy: the rider is meant to rule out self-report stuff like “Participants really enjoyed the session/thought it helped them”. I’d be happy to take things like “Participants got richer than controls”, “CFAR alums did better on these previously used metrics of decision making”, or whatever else.
Happy to discuss further to arrive at agreeable stipulations—or, if you prefer, we can just leave them to the judges discretion.
Ah, the 4-1 to one bet was a conditional one:
I don’t know CFAR’s current plans well enough to judge whether they will synthesize the relevant evidence. I am only betting that if they do, the result will be positive. I am still on the fence of taking a 4-1 bet on this, but the vast majority of my uncertainty here comes from what CFAR is planning to do, not what the result would be. I would probably take a 5-1 bet on the statement as you proposed it.
Sorry for misreading your original remark. Happy to offer the bet in conditional, i.e.:
Conditional on CFAR producing results of sufficient quality for academic publication (as judged by someone like Christiano or Karnofsky) these will fail to demonstrate benefit on a pre-specified objective outcome measure