I’ve been playing with causal inference lately, as one does.[1] I was thinking of writing a more formal sequence about how to do causal discovery and model comparison, and I might still do that.[2] Meanwhile, I’m going to start with a sort of informal diary of what I’m learning as I go.
I started a while ago, so this diary entry is coming partway into the process. I’ll go back and fill in the history if I write that formal sequence later. Here’s the little bit of backstory you need for today’s episode to make sense.
I had a data set about diabetes that I found in an ML data repository and I already played a lot with it over the last few weeks. I extracted a set of variables from it that seemed maybe causally connected and ran a bunch of analysis on them. In the end I was disappointed with that set; they all just seemed very entangled and I did not find any causal graph, or set of graphs, that seemed clearly better than the rest.
I started again with a different set of variables. This time, I picked ones that I believed were more likely to tell a story about the data—ideally, a story with at least one branch point, where the value of a variable would have downstream effects on other variables. Indeed, this time I did get clusters of causal graphs that fit the data far better than the rest, and in a way that made sense to me. Hurray!
Today’s tasks were to:
Refactor my code to work over any data set, with separate configuration, DAG-generation, DAG-scoring, and results-reporting modules;
Add some code to plot DAG fit over all possible DAGs, so I could see the clusters and identify what changed;
Create a synthetic test data set over a known DAG and run my code over it, to make sure everything was working right.
The code refactor was dull in the best possible way and went just fine. I won’t tell you about it.
Plotting DAG scores
When I started this morning, my code was printing out the total log evidence for the best causal model, and also showing me which edges in the DAG were contributing how much. It was also showing me the top 20 DAGs and their scores, so I had some sense for how the scores were falling off as I looked at worse causal models. The output looked like this:
================================================================================ BEST DAG ================================================================================ MED->MS; MED->PROC; MED->TIME; PD->MED; PD->MS; PD->PROC; PD->TIME; TIME->MS; TIME->PROC
Total log evidence: −325462.096600
Local score contributions by node: PD | parents=() score=-48378.393146 MS | parents=(‘MED’, ‘PD’, ‘TIME’) score=-78008.395650 TIME | parents=(‘MED’, ‘PD’) score=-67446.975997 MED | parents=(‘PD’,) score=-79906.821619 PROC | parents=(‘MED’, ‘PD’, ‘TIME’) score=-51721.510188
I didn’t love this output. I couldn’t really get a sense of what was going on there. I wanted to better understand the space of all causal models over each set of variables.
I decided to plot the log evidence over all of the models to see what that looked like. Here’s what it looked like for that first set of variables, the ones I suspected were pretty entangled.
I’m new at this so I’m still figuring out how to interpret what I’m looking at, but this sure looked like there was not much of a causal story here. Some DAGs were better than others but they were all in one big lump, smoothly degrading in score as the nodes got shuffled around.
Next I tried it on the second set of variables, the ones I expected to tell a much better causal story, and sure enough, the plot got way more interesting.
What we’re seeing here is that the first six thousand or so DAGs all did pretty well, but then that cluster was played out, and the next eight thousand or so DAGs all performed about the same as each other, and so on.
So… what did that first cluster have in common?
I added some analysis and labeling to my graph, shown there in red. DAGs in the best-performing cluster tended to have ATYPE and SOURCE as parents to MS, whereas DAGs in the second best cluster had MS without a parent. Also, in the first cluster, ATYPE tended to have MS and SOURCE as parents, whereas in the second cluster, ATYPE only had SOURCE as a parent.
Obviously this set of characteristics is mutually incompatible. These are DAGs. You can’t have ATYPE with MS as a parent and also MS with ATYPE as a parent. That’s fine; this is just analysis over all six-thousand top-scoring DAGs. What it’s really saying is that MS, ATYPE, and SOURCE were all entangled with each other and needed to be at the top of the causal graph together, and when you broke that relationship, you got worse DAGs. Versions of that same story were the largest factor in every step down. There were probably other issues, not involving the entanglement of those three variables, that explained why the steps were so far apart from each other, but the entanglement swamped the other issues and didn’t show up in the cluster labeling.
The next step here would be to throw away one or two of those three entangled variables and see what causal graphs we can get with only the remaining data, but I was happy enough with my DAG testing and scoring code at this point that I let it go and moved on.
Synthetic test data generation
Next up, I made a synthetic data set with a known DAG. The generator for my data set looks like this:
The first version of the story went like this.
I wake up in the morning and check my MOOD. If I feel social (50% chance), then I check if a FRIEND is available that day (40% chance). If there’s a friend, and if the WEATHER is snowy (10%) chance, then we do the ACTIVITY of skiing. If there’s a friend and it’s sunny (60% chance), then we have a picnic. If it’s rainy, or if there’s no friend, or if I feel focused, then I read books. Meanwhile, if it’s snowy, or 40% of the time when it’s rainy, I want to wear a SWEATER. And if I feel social then there’s an 80% chance of MUSIC, but if I feel focused then there’s only a 20% chance.
Then I generated 10k data points from that distribution, and tried to fit DAGs to the observed data to see what would happen.
Plotting DAGs for the synthetic data: skiing causes snow
My first run of this didn’t go great. It turned out that my data was too clean. A lot of cells were zero (you never see focused, sunny, and skiing together) and some had pretty small numbers (even social, friend available, and snowy is a rare combo). Having such a clean, deterministic data set caused a bunch of problems that I am not going to summarize here because it gets too into the weeds.
(Claude, my trusty assistant, seemed genuinely quite distressed that we were modeling the data as demonstrating that skiing caused snow. I thought for a while that it was wrong to be distressed, because some causal models are Markov-equivalent; e.g. there’s no way to tell if SWEATERs cause WEATHER or the other way around, in this data set, and that’s fine. But Claude was right to be a bit upset about it; the ACTIVITY part of the graph should in theory be fully recoverable.)
So I tried adding some noise to the data. I said that 90% of the time, I’d do the intended activity, but 10% of the time I’d pick one of the other two activities at random. Sometimes you just find yourself skiing in the rain even though you feel focused today, y’know?
This helped! Running with noisier data caused an almost-correct DAG to rise to the top—but I was unhappy with how close together the scores were. The fifth-scoring DAG postulated three extra edges between the independent variables of MOOD, WEATHER, and FRIEND, and I didn’t like to see that.
(As previously mentioned, the SWEATER/WEATHER and MOOD/MUSIC arrow flips are unavoidable.)
At this point, I was annoyed because I wanted to see a larger dropoff from the top four DAGs (all of which were as good as possible) to the next batch. I tried decreasing alpha (a parameter on the prior) by a bunch, effectively encouraging DAGS with fewer edges. This helped a little.
I could do more tuning, of course, but at some point I’m not sure what to port back from my testing setup to my real-world model comparison. Anything I do to the synthetic data (making more of it, making it noisier) doesn’t tell me much about whether the code is good. I’m not sure how or whether to port back the prior parameter tuning. I’m concerned about my real world data being quite sparse, and some of the same problems I saw on the test data set cropping up over there for the same reasons. And so on.
Especially if anyone asks me to! John wrote such a sequence about seven years ago, but it’s pretty brief and doesn’t go into a ton of detail. If anyone would get value out of having something more detailed, please say so.
Causal inference diary: skiing causes snow
I’ve been playing with causal inference lately, as one does.[1] I was thinking of writing a more formal sequence about how to do causal discovery and model comparison, and I might still do that.[2] Meanwhile, I’m going to start with a sort of informal diary of what I’m learning as I go.
I started a while ago, so this diary entry is coming partway into the process. I’ll go back and fill in the history if I write that formal sequence later. Here’s the little bit of backstory you need for today’s episode to make sense.
I had a data set about diabetes that I found in an ML data repository and I already played a lot with it over the last few weeks. I extracted a set of variables from it that seemed maybe causally connected and ran a bunch of analysis on them. In the end I was disappointed with that set; they all just seemed very entangled and I did not find any causal graph, or set of graphs, that seemed clearly better than the rest.
I started again with a different set of variables. This time, I picked ones that I believed were more likely to tell a story about the data—ideally, a story with at least one branch point, where the value of a variable would have downstream effects on other variables. Indeed, this time I did get clusters of causal graphs that fit the data far better than the rest, and in a way that made sense to me. Hurray!
Today’s tasks were to:
Refactor my code to work over any data set, with separate configuration, DAG-generation, DAG-scoring, and results-reporting modules;
Add some code to plot DAG fit over all possible DAGs, so I could see the clusters and identify what changed;
Create a synthetic test data set over a known DAG and run my code over it, to make sure everything was working right.
The code refactor was dull in the best possible way and went just fine. I won’t tell you about it.
Plotting DAG scores
When I started this morning, my code was printing out the total log evidence for the best causal model, and also showing me which edges in the DAG were contributing how much. It was also showing me the top 20 DAGs and their scores, so I had some sense for how the scores were falling off as I looked at worse causal models. The output looked like this:
I didn’t love this output. I couldn’t really get a sense of what was going on there. I wanted to better understand the space of all causal models over each set of variables.
I decided to plot the log evidence over all of the models to see what that looked like. Here’s what it looked like for that first set of variables, the ones I suspected were pretty entangled.
I’m new at this so I’m still figuring out how to interpret what I’m looking at, but this sure looked like there was not much of a causal story here. Some DAGs were better than others but they were all in one big lump, smoothly degrading in score as the nodes got shuffled around.
Next I tried it on the second set of variables, the ones I expected to tell a much better causal story, and sure enough, the plot got way more interesting.
What we’re seeing here is that the first six thousand or so DAGs all did pretty well, but then that cluster was played out, and the next eight thousand or so DAGs all performed about the same as each other, and so on.
So… what did that first cluster have in common?
I added some analysis and labeling to my graph, shown there in red. DAGs in the best-performing cluster tended to have ATYPE and SOURCE as parents to MS, whereas DAGs in the second best cluster had MS without a parent. Also, in the first cluster, ATYPE tended to have MS and SOURCE as parents, whereas in the second cluster, ATYPE only had SOURCE as a parent.
Obviously this set of characteristics is mutually incompatible. These are DAGs. You can’t have ATYPE with MS as a parent and also MS with ATYPE as a parent. That’s fine; this is just analysis over all six-thousand top-scoring DAGs. What it’s really saying is that MS, ATYPE, and SOURCE were all entangled with each other and needed to be at the top of the causal graph together, and when you broke that relationship, you got worse DAGs. Versions of that same story were the largest factor in every step down. There were probably other issues, not involving the entanglement of those three variables, that explained why the steps were so far apart from each other, but the entanglement swamped the other issues and didn’t show up in the cluster labeling.
The next step here would be to throw away one or two of those three entangled variables and see what causal graphs we can get with only the remaining data, but I was happy enough with my DAG testing and scoring code at this point that I let it go and moved on.
Synthetic test data generation
Next up, I made a synthetic data set with a known DAG. The generator for my data set looks like this:
The first version of the story went like this.
I wake up in the morning and check my MOOD. If I feel social (50% chance), then I check if a FRIEND is available that day (40% chance). If there’s a friend, and if the WEATHER is snowy (10%) chance, then we do the ACTIVITY of skiing. If there’s a friend and it’s sunny (60% chance), then we have a picnic. If it’s rainy, or if there’s no friend, or if I feel focused, then I read books. Meanwhile, if it’s snowy, or 40% of the time when it’s rainy, I want to wear a SWEATER. And if I feel social then there’s an 80% chance of MUSIC, but if I feel focused then there’s only a 20% chance.
Then I generated 10k data points from that distribution, and tried to fit DAGs to the observed data to see what would happen.
Plotting DAGs for the synthetic data: skiing causes snow
My first run of this didn’t go great. It turned out that my data was too clean. A lot of cells were zero (you never see focused, sunny, and skiing together) and some had pretty small numbers (even social, friend available, and snowy is a rare combo). Having such a clean, deterministic data set caused a bunch of problems that I am not going to summarize here because it gets too into the weeds.
(Claude, my trusty assistant, seemed genuinely quite distressed that we were modeling the data as demonstrating that skiing caused snow. I thought for a while that it was wrong to be distressed, because some causal models are Markov-equivalent; e.g. there’s no way to tell if SWEATERs cause WEATHER or the other way around, in this data set, and that’s fine. But Claude was right to be a bit upset about it; the ACTIVITY part of the graph should in theory be fully recoverable.)
So I tried adding some noise to the data. I said that 90% of the time, I’d do the intended activity, but 10% of the time I’d pick one of the other two activities at random. Sometimes you just find yourself skiing in the rain even though you feel focused today, y’know?
This helped! Running with noisier data caused an almost-correct DAG to rise to the top—but I was unhappy with how close together the scores were. The fifth-scoring DAG postulated three extra edges between the independent variables of MOOD, WEATHER, and FRIEND, and I didn’t like to see that.
(As previously mentioned, the SWEATER/WEATHER and MOOD/MUSIC arrow flips are unavoidable.)
At this point, I was annoyed because I wanted to see a larger dropoff from the top four DAGs (all of which were as good as possible) to the next batch. I tried decreasing alpha (a parameter on the prior) by a bunch, effectively encouraging DAGS with fewer edges. This helped a little.
I could do more tuning, of course, but at some point I’m not sure what to port back from my testing setup to my real-world model comparison. Anything I do to the synthetic data (making more of it, making it noisier) doesn’t tell me much about whether the code is good. I’m not sure how or whether to port back the prior parameter tuning. I’m concerned about my real world data being quite sparse, and some of the same problems I saw on the test data set cropping up over there for the same reasons. And so on.
This is a good place to stop for today.
Or at least, as one does, if one is in the process of metamorphosing into a technical alignment research, and one’s mentor is John Wentworth.
Especially if anyone asks me to! John wrote such a sequence about seven years ago, but it’s pretty brief and doesn’t go into a ton of detail. If anyone would get value out of having something more detailed, please say so.