I test 200mg caffeine in an n=1, m=50 self-blinded RCT. The outcomes are encouraging.
Log-score of predictions of substance
Absorption effect size d (λ, p, σ increase)
Mindfulness effect size d (λ, p, σ increase)
Productivity effect size d (λ, p, σ increase)
Creativity effect size d (λ, p, σ increase)
Happiness effect size d (λ, p, σ increase)
Contentment effect size d (λ, p, σ increase)
Relaxation effect size d (λ, p, σ increase)
Horniness effect size d (λ, p, σ increase)
200mg Caffeine (n=1, m=40)
-0.6
0.61 (λ=13.3, p=0.00017, −0.072)
0.58 (λ=11.8, p=0.0007, 0.021)
0.58 (λ=28.9, p=1.3-12, 0.11)
0.38 (λ=32.9, p=5.2-15, 0.09)
0.27 (λ=10.6, p=0.002, 0.3)
0.13 (λ=7.66, p=0.02, 0.47)
-0.11 (λ=5, p=0.15, 0.42)
-0.14 (λ=1.9, p=0.64, 0.11)
I am especially interested in testing many different substances for their effect on meditation, while avoiding negative side effects. The benefits from high meditational attainments valuable to me, and seem especially likely to benefit from chemical intervention, since the Algernon argument likely doesn’t apply: Meditative attainments might’ve not led to a fitness advantage (even, by opportunity cost, to a fitness disadvantage), and so were likely selected against, but most of us don’t care that much about inclusive genetic fitness and more about psychological well-being. Evolutionary dynamics favor being like Dschingis Khan (dozens to hundreds of offspring) over Siddharta Gautama (one son), but I’d rather attain sotāpanna than pillage and murder.
And meditative attainments are costly: they take tens to hundreds to thousands of hours to reach, which would make simple psychopharmacological interventions worthwhile. I also don’t buy that they miss the point of meditation—most people already struggle enough, so some help doesn’t make it a cakewalk; “reach heaven through fraud”. One must be careful not to fall into the trap of taking substances that feel good but lessen sensory clarity (which I believe was the original intent behind the fifth precept, and so I’ll exclude e.g. opiates from the substances to test.
Meditation: 45 minutes of ānāpānasati, started 0-60 minutes after taking the dose, tracking two variables.
Mindfulness: How aware I was of what was going on in my head, modulo my ability to influence it.
Absorption (often called concentration): How “still” my mind was, how easily I was swept away by my thoughts.
Flashcard performance: Did my daily flashcards for ~20 minutes, started 0-60 minutes after finishing meditation.
Arm Prediction: I tried to predict whether the substance I’d taken was placebo or caffeine.
Mood: Tracking 4 different variables at random points during the day, namely
Happiness/Sadness
Contentment/Discontentment
Relaxation/Stress
Horniness/Chastity: Chastity being simply the opposite of horniness in this case.
Productivity and creativity, recorded at the end of the day.
The total cost of the experiment is at least 21.5€:
Time: The Clearer Thinking tool for the value of my time returns 15€/hour, which gives a time cost of 18.75€ for preparing the experiment.
Time for filling: 35 minutes
Time for preparing envelopes: 40 minutes
Cost of caffeine pills: 0.0825€200mg caffeine pill⋅ 200mg caffeine pills=2.0625€
Cost of empty capsules: 0.03€capsule⋅25 capsules=0.75€
Cost of sugar: Negligible.
200mg caffeine pills, placebo pills filled with sugar, of each 25. Put each pill with a corresponding piece of paper (“C” for caffeine, “P” for placebo) into an unlabeled envelope. Used seq 1 50 | shuf to number the envelopes, and sorted them accordingly.
Notes on the experiment:
3rd dose: Out of fear that the placebo pills have some sugar stuck outside of them, which could de-blind the dose, I take a bit (~10 g) of sugar with each pill.
7th dose: Increase time between consumption and starting to meditate to ~45 minutes, after finding out that the onset of action is 45 minutes-1 hour.
14th dose: Noticed that during meditation, sharpness/clarity of attention is ~high, and relaxing after becoming mindful is easy, but attention strays just as easily.
49th dose: Took the pill, meditated, lay down during meditation and fell asleep. Likely10% placebo.
Statistical Method
In general, I’ll be working with the likelihood ratio test. For this, let vP be the distribution of values of a variable for the placebo arm, and vC the distribution of values for a variable of the caffeine arm. (I apologise for the C being ambiguous, since it could also refer to the control arm).
Then let θ0=(μ0,σ0)=MLEN(vP) be the Gaussian maximum likelihood estimator for our placebo values, and θ=(μ,σ)=MLEN(vC) be the MLE for our caffeine values.
Then the likelihood ratio statistic λ is defined as λ=2logLC(θ)LC(θ0) where LC(θ) is the likelihood the caffeine distribution assigns to the parameters θ. This test is useful here because we fix all values of θ0. See Wasserman 2003 ch. 10.6 for more.
If λ≈0, then the MLE for the placebo arm is very close to the MLE for the caffeine arm, the distributions are similar. If λ>0, then the MLE for the placebo arm is quite different from the caffeine arm (though there is no statement about which has higher values). λ<0 is not possible, since that would mean that the MLE of the placebo distribution has a higher likelihood for the caffeine data than the MLE of the caffeine distribution itself—not very likely.
Note that I’m not a statistician, this is my first serious statistical analysis, so please correct me if I’m making some important mistakes. Sorry.
Predictions on the Outcomes of the Experiment
After collecting the data, but before analysing it, I want to make some predictions about the outcome of the experiment, similar to another attempt here.
Some unimportant pre-processing, in which we filter for mood recordings 0-10 hours after caffeine intake, since pd.merge_asof doesn’t do cartesian product:
Which leads to d of ~0.27 for happiness, ~0.13 for contentment, ~-0.11 for relaxation and ~-0.14 for horniness.
Likelihood Ratios
We assume (at first) that the data is distributed normally. Then we can define a function for the gaussian likelihood of a distribution given some parameters:
And now we can compute the likelihood ratio LθLθ0 for the null hypothesis θ0=MLE(vP) for the placebo data vP, and also the result of the likelihood ratio test:
Caffeine appears helpful for everything except relaxation (and it makes me hornier, which I’m neutral about). I’d call this experiment a success and will be running more in the future, while in the meantime taking caffeine before morning meditations.
Predicting the outcomes of personal experiments give a useful way to train ones own calibration, I take it a step further and record the predictions for the world to observe my idiocy.
Prediction of Arm
My prediction about the content of the pill is more accurate than random guesses80%: Yes.
My prediction about the content of the pill has a log score of more than −0.560%: No.
Meditation
On days with caffeine, my average mindfulness during meditation was higher than days with placebo60%: Yes.
On days with caffeine, my average absorption during meditation was higher than days with placebo40%: Yes.
On days with caffeine, the variance of values for mindfulness during meditation was lower than on placebo days55%: No.
On days with caffeine, the variance of values for absorption during meditation was lower than on placebo days35%: Yes.
Self-Blinded Caffeine RCT
I am especially interested in testing many different substances for their effect on meditation, while avoiding negative side effects. The benefits from high meditational attainments valuable to me, and seem especially likely to benefit from chemical intervention, since the Algernon argument likely doesn’t apply: Meditative attainments might’ve not led to a fitness advantage (even, by opportunity cost, to a fitness disadvantage), and so were likely selected against, but most of us don’t care that much about inclusive genetic fitness and more about psychological well-being. Evolutionary dynamics favor being like Dschingis Khan (dozens to hundreds of offspring) over Siddharta Gautama (one son), but I’d rather attain sotāpanna than pillage and murder.
And meditative attainments are costly: they take tens to hundreds to thousands of hours to reach, which would make simple psychopharmacological interventions worthwhile. I also don’t buy that they miss the point of meditation—most people already struggle enough, so some help doesn’t make it a cakewalk; “reach heaven through fraud”. One must be careful not to fall into the trap of taking substances that feel good but lessen sensory clarity (which I believe was the original intent behind the fifth precept, and so I’ll exclude e.g. opiates from the substances to test.
Variables tracked (see more here):
Meditation: 45 minutes of ānāpānasati, started 0-60 minutes after taking the dose, tracking two variables.
Mindfulness: How aware I was of what was going on in my head, modulo my ability to influence it.
Absorption (often called concentration): How “still” my mind was, how easily I was swept away by my thoughts.
Flashcard performance: Did my daily flashcards for ~20 minutes, started 0-60 minutes after finishing meditation.
Arm Prediction: I tried to predict whether the substance I’d taken was placebo or caffeine.
Mood: Tracking 4 different variables at random points during the day, namely
Happiness/Sadness
Contentment/Discontentment
Relaxation/Stress
Horniness/Chastity: Chastity being simply the opposite of horniness in this case.
Productivity and creativity, recorded at the end of the day.
The total cost of the experiment is at least 21.5€:
Time: The Clearer Thinking tool for the value of my time returns 15€/hour, which gives a time cost of 18.75€ for preparing the experiment.
Time for filling: 35 minutes
Time for preparing envelopes: 40 minutes
Cost of caffeine pills: 0.0825€200mg caffeine pill⋅ 200mg caffeine pills=2.0625€
Cost of empty capsules: 0.03€capsule⋅25 capsules=0.75€
Cost of sugar: Negligible.
200mg caffeine pills, placebo pills filled with sugar, of each 25. Put each pill with a corresponding piece of paper (“C” for caffeine, “P” for placebo) into an unlabeled envelope. Used
seq 1 50 | shuf
to number the envelopes, and sorted them accordingly.Notes on the experiment:
3rd dose: Out of fear that the placebo pills have some sugar stuck outside of them, which could de-blind the dose, I take a bit (~10 g) of sugar with each pill.
7th dose: Increase time between consumption and starting to meditate to ~45 minutes, after finding out that the onset of action is 45 minutes-1 hour.
14th dose: Noticed that during meditation, sharpness/clarity of attention is ~high, and relaxing after becoming mindful is easy, but attention strays just as easily.
49th dose: Took the pill, meditated, lay down during meditation and fell asleep. Likely10% placebo.
Statistical Method
In general, I’ll be working with the likelihood ratio test. For this, let vP be the distribution of values of a variable for the placebo arm, and vC the distribution of values for a variable of the caffeine arm. (I apologise for the C being ambiguous, since it could also refer to the control arm).
Then let θ0=(μ0,σ0)=MLEN(vP) be the Gaussian maximum likelihood estimator for our placebo values, and θ=(μ,σ)=MLEN(vC) be the MLE for our caffeine values.
Then the likelihood ratio statistic λ is defined as λ=2logLC(θ)LC(θ0) where LC(θ) is the likelihood the caffeine distribution assigns to the parameters θ. This test is useful here because we fix all values of θ0. See Wasserman 2003 ch. 10.6 for more.
If λ≈0, then the MLE for the placebo arm is very close to the MLE for the caffeine arm, the distributions are similar. If λ>0, then the MLE for the placebo arm is quite different from the caffeine arm (though there is no statement about which has higher values). λ<0 is not possible, since that would mean that the MLE of the placebo distribution has a higher likelihood for the caffeine data than the MLE of the caffeine distribution itself—not very likely.
Note that I’m not a statistician, this is my first serious statistical analysis, so please correct me if I’m making some important mistakes. Sorry.
Predictions on the Outcomes of the Experiment
After collecting the data, but before analysing it, I want to make some predictions about the outcome of the experiment, similar to another attempt here.
Moved here.
Analysis
We start by setting everything up and loading the data.
The mood data is a bit special, since it doesn’t have timezone info, but that is easily remedied.
Summary Statistics
We can first test how well my predictions fared:
drumroll
At least this time I was better than chance:
Meditation
Merging the meditations closest (on the right) to the consumption and selecting the individual variables of interest:
So, does it help?
Indeed! Cohen’s d here looks pretty good. Taking caffeine also reduces the variance of both variables:
Productivity and Creativity
We repeat the same procedure for the productivity and creativity data:
And the result is…
Again surprisingly good! The creativity values are small enough to be a fluke, but the productivity values seem cool.
In this case, though, caffeine increases variance in the variables (not by very much):
Mood
Some unimportant pre-processing, in which we filter for mood recordings 0-10 hours after caffeine intake, since
pd.merge_asof
doesn’t do cartesian product:And now the analysis:
Which leads to d of ~0.27 for happiness, ~0.13 for contentment, ~-0.11 for relaxation and ~-0.14 for horniness.
Likelihood Ratios
We assume (at first) that the data is distributed normally. Then we can define a function for the gaussian likelihood of a distribution given some parameters:
And now we can compute the likelihood ratio LθLθ0 for the null hypothesis θ0=MLE(vP) for the placebo data vP, and also the result of the likelihood ratio test:
And this gives us surprisingly large values:
And, if one is interested in p-values, those correspond to (with 2 degrees of freedom each):
I find these results surprisingly strong, and am still kind of mystified why. Surely caffeine isn’t that reliable!
And, the same, for mood:
And the p-values of those are:
Conclusion
Caffeine appears helpful for everything except relaxation (and it makes me hornier, which I’m neutral about). I’d call this experiment a success and will be running more in the future, while in the meantime taking caffeine before morning meditations.
Full code for the experiment here.
Predictions on the Outcome of the Experiment
Predicting the outcomes of personal experiments give a useful way to train ones own calibration, I take it a step further and record the predictions for the world to observe my idiocy.
Prediction of Arm
My prediction about the content of the pill is more accurate than random guesses80%: Yes.
My prediction about the content of the pill has a log score of more than −0.560%: No.
Meditation
On days with caffeine, my average mindfulness during meditation was higher than days with placebo60%: Yes.
On days with caffeine, my average absorption during meditation was higher than days with placebo40%: Yes.
On days with caffeine, the variance of values for mindfulness during meditation was lower than on placebo days55%: No.
On days with caffeine, the variance of values for absorption during meditation was lower than on placebo days35%: Yes.
λ<1 for the mindfulness values20%: No.
λ<1 for the absorption values25%: No.
λ<4 for the mindfulness values82%: No.
λ<4 for the absorption values88%: No.
Mood
On days with caffeine, my average happiness during the day was higher than days with placebo65%: Yes.
On days with caffeine, my average contentment during the day was higher than days with placebo45%: Yes.
On days with caffeine, my average relaxation during the day was higher than days with placebo35%: No.
On days with caffeine, my average chastity during the day was higher than days with placebo50%: No.
On days with caffeine, the variance of values for happiness during the day was lower than on placebo days55%: No.
On days with caffeine, the variance of values for contentment during the day was lower than on placebo days30%: No.
On days with caffeine, the variance of values for relaxation during the day was lower than on placebo days30%: No.
On days with caffeine, the variance of values for chastity during the day was lower than on placebo days50%: No.
λ<1 for the happiness values45%: No.
λ<1 for the contentment values40%: No.
λ<1 for the relaxation values37%: No.
λ<1 for the chastity values60%: No.
λ<4 for the happiness values85%: No.
λ<4 for the contentment values90%: No.
λ<4 for the relaxation values90%: No.
λ<4 for the chastity values95%: Yes.
Productivity and Creativity
On days with caffeine, my average productivity during the day was higher than days with placebo52%: Yes.
On days with caffeine, my average creativity during the day was higher than days with placebo55%: Yes.
On days with caffeine, the variance of values for productivity during the day was lower than on placebo days40%: No.
On days with caffeine, the variance of values for creativity during the day was lower than on placebo days65%: No.
λ<1 for the productivity values40%: No.
λ<1 for the creativity values45%: No.
λ<4 for the productivity values75%: No.
λ<4 for the creativity values80%: No.
I also recorded my predictions about the content of the pill on PredictionBook (1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50).