Toward “timeless” continuous-time causal models

I’m a bit at a loss as to where to put this. I know the in­fer­en­tial gap is too great for it to go any­where but here, and I know that the num­ber of peo­ple on LW in­ter­ested in this sub­ject could be counted on one hand. The pre­req­ui­sites would al­most cer­tainly be Time­less Causal­ity and more math­e­mat­ics than any­one is re­ally in­ter­ested in learn­ing.

So, I apol­o­gize in ad­vance if you read this and dis­cover at the end it was a waste of your time. But at the same time, I need peo­ple who know about these things to talk about them with me, to en­sure that I haven’t gone crazy… yet. And most im­por­tantly, I need to know the peo­ple who have done this be­fore, so that I don’t have to do it. Google can’t find them.


There are cur­rently some efforts to gen­er­al­ize the causal mod­els of Pearl to con­tin­u­ous-time situ­a­tions. Most of these at­tempts in­volve re­plac­ing some causal dis­crete vari­ables Xi with time-de­pen­dent ran­dom vari­ables Xi(t). Pos­si­bly due to memetic in­fec­tion from Yud­kowsky, I don’t think this is nec­es­sar­ily the cor­rect ap­proach. The philo­soph­i­cal power of Pearl’s the­ory comes from the fact that it is time­less, that ba’o vimcu ty bu.

In or­der to mo­ti­vate my work­ing defi­ni­tions for where such a time­less con­tin­u­ous-time the­ory will go, I need to go back to clas­si­cal causal­ity and de­cide what a time­less for­mu­la­tion ac­tu­ally means, for­mally. Spoiler: it means re­plac­ing time-de­pen­dent evolu­tion with a global flow on the phase space of the sys­tem. This is more or less in line with what is said in Time­less Physics with re­gard to the glimpse of “quan­tum mist” illus­trated there.

The role of phase space

What is “time­less­ness”? The first thing I thought of af­ter read­ing the time­less sub­se­quence was, “What does a time­less for­mu­la­tion of the wave equa­tion look like?” First of all, this was the right thought, be­cause the wave equa­tion is what I’ll call (af­ter the fact) “clas­si­cally causal” in a sense to be de­scribed soon. I wouldn’t have seen the time­less­ness in a differ­ent math­e­mat­i­cal model, be­cause not all math­e­mat­i­cal mod­els of re­al­ity pre­serve the un­der­ly­ing phe­nom­ena’s causal struc­ture. On the other hand, this was the wrong thought, be­cause the wave equa­tion is not the sim­plest con­tin­u­ous-time sys­tem that would have led me to this for­mal­iza­tion of time­less­ness. Un­for­tu­nately the one that is eas­ier for me to see (La­grangian me­chan­ics) is harder for me to ex­plain, so you’re stuck with a sub­op­ti­mal ex­pla­na­tion.

The wave equa­tion mod­els all sorts of wave-like phe­nom­ena: light, acous­tic waves, earth­quakes, and so on. If we take the speed of sound to be one (as physi­cists are wont to do), the dis­per­sion re­la­tion is ω2 = k2. Such a dis­per­sion re­la­tion satis­fies the Kramers-Kronig re­la­tion. As it turns out, equa­tions whose dis­per­sion re­la­tion satis­fies this con­di­tion satisfy what I’m call­ing “clas­si­cal causal­ity”, but what is more com­monly known as finite speed of prop­a­ga­tion — or, more phys­i­cally speak­ing, the fact that sig­nals stay within their light cone.

The most com­mon prob­lem as­so­ci­ated with the wave equa­tion is the Cauchy prob­lem. At time zero, we spec­ify the state of the sys­tem: its ini­tial po­si­tion and ve­loc­ity at ev­ery point. Then the solu­tion of the wave equa­tion de­scribes how that ini­tial state evolves with time. From a more ab­stract point of view, this evolu­tion is a curve in the space of all pos­si­ble ini­tial states. This space is com­monly referred to in the spe­cific case of the wave equa­tion as “en­ergy space”, which fur­ther illus­trates why this ex­am­ple is a bit bad for ped­a­gog­i­cal pur­poses. From now on, we’re only go­ing to talk about phase space.

Here is where we can re­move time from the equa­tion. In­stead of think­ing of the wave equa­tion as as­so­ci­at­ing to ev­ery state in phase space a time-de­pen­dent curve is­su­ing forth from it, we’re go­ing to think of the wave equa­tion as spec­i­fy­ing a global flow on the whole of phase space, all at once. In sum­mary, I am led to be­lieve that time­less for­mu­la­tions amount to ab­stract­ing away the time-de­pen­dence of the sys­tem’s evolu­tion as a flow on the phase space of the sys­tem. And to think, this in­sight only took three years to in­ter­nal­ize, pro­vided I’ve got­ten it cor­rect.

Causal flow

The situ­a­tion for a causal model is harder. In part, be­cause stochas­tic things have shoddy ex­cuses for deriva­tives. For the mo­ment, we’re go­ing to take the eas­iest pos­si­ble con­tin­u­ous-time sys­tem: our causal N vari­ables of in­ter­est, Xi, take only real val­ues. The space of all the pos­si­ble states of the sys­tem is N-di­men­sional Eu­clidean space, which is easy enough to work with. I’m go­ing to im­plic­itly as­sume that causal vari­ables evolve con­tin­u­ously; that is, the side­walk doesn’t go from be­ing com­pletely dry to com­pletely wet in­stan­ta­neously. Things like light switches and push but­tons can still be mod­eled prac­ti­cally by bump func­tions and the like, so I don’t see this as a real limi­ta­tion.

The some­what harder bul­let to swal­low is the as­sump­tion that the ran­dom vari­ables are Marko­vian; that is, they are “mem­o­ryless” in the sense that only the pre­sent state de­ter­mines the fu­ture. Pearl spends some time in Causal­ity defend­ing this as­sump­tion from crit­i­cism that it doesn’t ap­ply to quan­tum sys­tems — I be­lieve this defense is rea­son­able. I be­lieve that causal mod­els are nec­es­sar­ily re­fine­ments of our be­liefs about what is still for the most part a clas­si­cal world, and so the Markov as­sump­tion is not nec­es­sar­ily un­nat­u­ral.

The phase space of N-di­men­sional Eu­clidean space is known as the tan­gent bun­dle, which amounts to hav­ing an ad­di­tional copy of N-space at ev­ery point. Mo­rally speak­ing, the tan­gent bun­dle rep­re­sents all the di­rec­tions and speeds in which the sys­tem can evolve from any given state.

We need some data about how the sys­tem is sup­posed to evolve: what I will call the causal flow. As best as I can cur­rently con­jec­ture, this data should take the form of a “bun­dle” of prob­a­bil­ity mea­sures P, one for each point in N-space, such that each prob­a­bil­ity mea­sure P(x) is defined over the tan­gent copy of N-space at­tached to that point.

By anal­ogy with the pre­vi­ous sec­tion, the time-evolu­tion of the sys­tem is given by Lip­s­chitz-con­tin­u­ous curves in N-space. (Lip­s­chitz-con­tin­u­ous, be­cause if we as­sume they are differ­en­tiable curves, the Markov as­sump­tion goes out the win­dow.) In con­trast with the dis­crete the­ory of causal­ity, and as men­tioned above, we don’t al­low causal vari­ables to “jump” spon­ta­neously, and there is a limit to how sharply they can “turn”.

A use­ful thing to have around would be the prob­a­bil­ity that the sys­tem will evolve from one state to an­other via a spe­cific choice of one of these curves. Lip­s­chitz-con­tin­u­ous curves are rec­tifi­able, and so one can re­ca­pitu­late a sort of Rie­mann sum — if you’re in­ter­ested, I have it for­mally writ­ten down in a .pdf, but the cur­rent for­mat is un­friendly to maths. So for now, you’ll just have to take my word for it when I say I can define the prob­a­bil­ity of the flow fol­low­ing a spe­cific path. From there, it’s just a path in­te­gral to defin­ing the prob­a­bil­ity of get­ting from one state to an­other.

Where to go from here?

Given this causal flow, d-sep­a­ra­tion should arise as a ge­o­met­ri­cal con­di­tion — but per­haps only a lo­cal one, for the causal struc­ture of the sys­tem can also evolve with time. To in­ter­vene in this sys­tem is to pro­ject it onto a cer­tain hy­per­plane, pre­sum­ably, in some yet-to-be-de­ter­mined way. And fi­nally, there ought to be some way to define coun­ter­fac­tu­als, but my limited math­e­mat­i­cal fore­sight has already run too thin.

BONUS: If you’ve made it this far and can’t think of any­thing else to say, I’m will­ing to Crocker-en­ter­tain prob­a­bil­ities that I’m in­sane and/​or a crack­pot.