I have a new project for which I actively don’t want funding for myself: it’s too new and unformed to withstand the pressure to produce results for specific questions by specific times*. But if it pans out in ways other people value I wouldn’t mind retroactive payment. This seems like a good fit for impact certificates, which is a tech I vaguely want to support anyway.
Someone suggested that if I was going to do that I should mint and register the cert now, because that norm makes IC markets more informative, especially about the risk of very negative projects. This seems like a good argument but https://www.impactcerts.com/mint seems borked and I don’t know of better options. Now this is requiring thought and the whole point was to not have to do that yet.
So I’m crowdsourcing. What are your thoughts on this? What are potential best practices I should support? Counter arguments?
*my psychology is such that there’s no way around this that also guarantees speeding up the work. If someone wanted to fund the nice things for Elizabeth project I’d accept but no guarantee I would produce any faster. I *have* asked for funding for my collaborator and a TBD research assistant.
I will definitely not be sharing the object level project in this thread.
I hurt my hand so if my replies look weird that’s why.
Elizabeth
Elizabeth’s Shortform
Can you expand on this?
See discussion here. If someone wanted to make the investment in a frequently updating system I have ideas for how to do it, but I don’t plan on making it myself.
I’ve known several men who had sexual encounters with women that… labeling them is hard, let’s say the encounters left them unhappy, and would have been condemned if the sexes had been reversed. These men encountered a damaging amount of pushback and invalidation when they tried to discuss their feelings about those encounters. One was literally told “I hope you were grateful”, for others the invalidation was more implicit. For at least 2, me saying “that sounds fucked up” and then listening was an extremely helpful novelty. So I’m really nervous about pushing the wider cultural narrative in ways that would reduce the ability of male victims to be upset.
OTOH, one of those dudes had his complicated experiences with a same-age girlfriend, and had nothing but good things to say about losing his virginity in high school to a woman twice his age. You definitely couldn’t help him by creating and enforcing general rules about what counts as bad.
But people try lots of weird fad diets. Moreover popular wisdom seems to be capable of having different reactions to different fad diets- grapefruit diet is considered dangerous and you’ll regain the weight immediately anyway, but keto is considered to sometimes work if you can stick to it, which is pretty hard. It’s not clear to me keto started off obviously distinguishable from other fad diets of the time- but it worked for some people and was sustainable for some people, and so grew in prominence,
TBC I don’t think eating only potatoes is long term healthy. If the weak form of potato diet- where you eat some potatoes every day but are otherwise unrestricted- works, that is substantially easier to follow and less weird than a lot of popular diets people stay on for years.
I am highly skeptical because that’s the correct default stance to have towards monodiets claiming dramatic levels of weight loss, but if it worked in a healthy sustainable way I don’t expect weirdness to kill it.
Cognitive Risks of Adolescent Binge Drinking
I think I used “silver bullet” to use the same thing you mean by “$20 bill”, so we’re in broad agreement. I also don’t personally know anyone who thought this was definitely a slam dunk; everyone I’ve talked to has had the attitude “sounds crazy, generally good to try things”.
I agree with you that the weight loss may be too rapid to be healthy, and that the data is basically worthless without knowing what the rebound rate is. I also dislike the emphasis on weight loss over 24 hours, when it’s impossible to have one-day weight loss that is both noticeable above the noise in the measurement and healthy.
I disagree that it being prohibitively restrictive for many people is a reason not to investigate. The restrictiveness and social costs aren’t secret harms people won’t notice until it’s too late; people will naturally notice those costs and change their behavior if it’s not worth it. SMTM claims the diet tolerates a lot of deviance, so the costs may be quite low. Maybe that slows the weight loss, but people can make their own choices on that. It seems much more forgiving than keto, where one carb too many breaks the diet for days, and despite a very high attrition rate there’s a substantial number of people sticking to keto long term. The high attrition rate is irrelevant to knowing if the diet works when you stick to it.
I would feel differently if they were charging a large up front fee and blaming people for not sticking to it, but that is not at all happening. They’re suggesting people eat a very cheap food and stop if they don’t like it. This might change if potato fat camp happens, at which point I do hope they highlight the drop out rate, but I really don’t see “you might quit” as a reason not to try.
I also disagree that we should wait for drugs. Those are definitely worth investigating, but the history of weight loss drugs and especially drugs in trials is really bad, not to mention none of them are as widely available as potatoes.
I still think the potential silent risks are a reason to be concerned. I doubt their reports that potatoes have enough protein, especially for highly active people, and expect micronutrient shortages as well. I hope a more formal study checks those. I didn’t try the diet because there was absolutely no chance “100% potatoes” would be healthy for me, and I advised the friend who asked to wait on the data. I put an extremely large probability on “this is just another fad monodiet and it’s bad in the ways they are all bad”. But trying it out, especially in the extremely flexible way they are, still seems good to me.
Part of me strongly agrees with you. At the same time… I don’t think “how much work would it have been to address [single bad thing]?” is the right unit of measurement. The actual question is “what would the cost/benefit analysis of address every single thing in this class of problems?”, and that’s a lot less clear cut.
Some things that make it difficult:
There are just a lot of potential problems.
There are an infinite number of things they didn’t research. Every time they add one to the explicit list of unresearched things they make the list as a whole harder to read (-> people are more likely to miss warnings relevant to them) while also conveying more safety and confidence. God forbid people decide anything not on the list of “things we didn’t check” was in fact checked.
You can say they should estimate the danger from each of these and only explicitly disavow research over a certain level of importance, but that’s an inherently noisy process people are likely to disagree on, and no matter what there has to be something just below the line. So again adding warnings can leave people with an inaccurately increased feeling of safety.
When I share medical info, I always preface it with “anything with a real effect can hurt you, think this through for yourself”. I think that’s a little better than what SMTM did. But I don’t see any easy additions to their statements of “If at any point you get sick or begin having side-effects, stop the diet immediately. We can still use your data up to that point, and we don’t want anything to happen to you” and “We are not doctors. We are 20 rats in a trenchcoat. eee! eee! eee!” that make things obviously better.
You note that you felt obliged to keep going in order to provide better data. Maybe there was pressure in private communication, but AFAICT they explicitly spoke against that in the blog post. I had a similar thing happen in my experiment- despite explicitly saying people could drop out at any time and would still be paid, one person was extremely reluctant to do so despite bad side effects and needed a strong push from me. In some sense I don’t think this is fair to put on experimenters, people shouldn’t even need to be told “stop if you feel bad”. But this seems like a pattern, and therefore should be taken into account even if it’s not objectively fair.
I wouldn’t have made participation mandatory (and I expect you wouldn’t either) but a way for participants to opt-in to a community does seem valuable. Maybe they thought that was covered by twitter? A lot of people did seem to be discussing it there.
This seems useful and I liked the author’s book but I wouldn’t call it “well-understood reasons that are entirely accounted for by mainstream obesity science”. If anything, the fact that we’ve known about it for >10 years and it hasn’t spread widely suggests to me that it’s unlikely to be a silver bullet.
What data do you think they should be sharing that they aren’t? They did put a dataset up somewhere.
On one hand, I’ve been disappointed with SMTM’s handling of this study. I would rather have had more caveats, more pushes to make people more responsible for their own health, and more contextualization around crash diets and monodiets.
On the other hand, I want this data. And I want more experiments in general. When I tried to run my own study, for a much smaller ask than a monodiet, with what I view as acceptable amounts of caveats and contextualization, I really struggled with recruitment and subject follow through. That’s partially a function of audience size, except my smaller audience is in part downstream of me being more cautious and less sweeping in my claims in general. I’m just less fun to read than Slime Mold Time Mold.[it didn’t help follow through that I was specifically recruiting people with ADHD, who are hell’s own subject group, but the issue with recruitment was audience size and excitement]
This doesn’t undo any of the damage done by SMTM’s lack of caution (it feels unfair judging them on your specific case because they did tell you not to do it, but I’d independently noticed less caution and caveats than I’d like). But if we want this kind of data at all we either need to accept those costs, or become more willing to participate in experiments with accurate caveats.
it worked for Jon Snow I don’t know what your problem is.
To the list of actions I want to add: don’t use commitments to extend the reach of your social pressure.
There’s a trap where people ask for ambiguous commitments in ways that are socially awkward to say no to (because obviously they mean the weak version that only a mean or austic person would say no to!), and then try to hold you to the strong version later (because you committed!).
I’m really curious if their count of debt included money owed but not yet tracked by credit bureaus (e.g. modestly late utilities or medical payments), informal loans (e.g. from family), or informal loans the recipient had perhaps given up on until they found out you had money).
EDIT: interesting discussion of this on twitter, getting into situations that could never have been tracked as debt per se, but in the end are claims on your money by other people that your extended social network will view as legitimate and punish you if you don’t fill (and that punishment is worse than losing the money). I think an individual or nuclear family might just be the wrong unit of intervention or measurement.
That’s only true if people within states are more similar to each other on the relevant axes than to people in other states, right? If the real divide is rural/urban or education, then comparing states isn’t very useful even if some states are more rural or educated than others.
The fact that the county-level data is bad is unfortunate and makes the county-level analysis less useful, but doesn’t fix any of the problems with state-level data.
State average altitude as a proxy for individual altitude, as a proxy for altitude of water source, as a proxy for water contamination. Jury is out on correctness.
(I’m broadly on board with contaminant theory of obesity but state is never the right level of data to look at except for laws).
Nuclear power plants need to be a pretty specific distance from water, which hurts their use as a solution to a problem that causes rising water levels