The archetypal example for this is something like a thermostat. The variable S represents random external temperature fluctuations. The regulator R is the thermostat, which measures these fluctuations and takes an action (such as putting on heating or air conditioning) based on the information it takes in. The outcome Z is the resulting temperature of the room, which depends both on the action taken by the regulator, and the external temperature.
The ordinary room thermostat does not measure S. It measures Z. Its actions are determined by Z and the reference temperature set by the user (in your example assumed constant and omitted for simplicity). There is no causal arrow from S to R. Here is the true causal diagram:
The room thermostat is therefore outside the scope of the GRT. This is true of every control system that senses the variable it is controlling, i.e. every control system.
The diagram is a causal Bayes net which is a DAG so it can’t contain cycles. Your diagram contains a cycle between R and Z. The diagram I had in mind when writing the post was something like:
which is a thermostat over a single timestep.
If you wanted to have a feedback loop over multiple timesteps, you could conjoin several of these diagrams:
Each node along the top row is the temperature at successive times. Each node along the bottom row is the controller state at different times.
The diagram is a causal Bayes net which is a DAG so it can’t contain cycles. Your diagram contains a cycle between R and Z.
Indeed. So much the worse for causal Bayes nets. People have tried to extend Pearl-type causal analysis to cyclic causal graphs, but have not got very far. The limits of our tools are not the limits of the world.
I am not convinced by the idea of unrolling the graph in time. Outside of digital electronics, time is continuous [1]. There is no time step for the room thermostat. For more complicated systems, the causal relationships you see when you discretise time may vary depending on the chosen time step. Continuous time and differential equations are the right abstraction.
Outside of digital electronics, time is continuous
Oh, I see the misunderstanding! So, what you can do here is fix the issue by taking the limit as the time step goes to 0, or equivalently, saying (Temperature) directly affects both and (Regulator), while affects .
What we need to do here is remember that both and are (possibly continuous) functions of (time), and think of these as showing at one point in time. The important fact about this system is can’t affect directly/instantaneously (which violates causality), but it can affect (or in discrete time, ). For example, your thermostat might control how much electricity travels through a coil to heat your room. However, it can’t instantly bump the temperature up by 20 degrees. This would be a logical contradiction— clearly can’t force to be 20 degrees higher than… . just doesn’t work.
The unrolled loop still should not have the arrow between S and R, and should have the feedback, because the controller is sensing the outcome. Unrolled it should go: S1 → Z1, R1->Z1, Z1 → R2, R2 → Z2, S2-> Z2 …
How do you figure a thermostat directly measures what it’s controlling? It controls heat added/removed per unit time, typically just more/less/no change, and measures the resulting temperature at a single point on typically a minute+ delay due to the dynamics of the system (air and heat take time to diffuse, even with a blower). Any time step sufficiently shorter than that delay is going to work the same. The current measurement depends on what the thermostat did tens of seconds if not minutes previously.
There are times the continuous/discrete distinction is very important but this example isn’t one of them. As soon as you introduce a significant delay between cause and effect the time step model works (it may well be a dependence on multiple previous time-steps, but not the current one).
I don’t think this is an unusual example, we have a small number of sensors, we get data on a delay, and we’re actually trying to control e.g. the temperature in the whole house, holding a set point, minimizing variation between rooms and minimizing variation across time, with the smallest amount of control authority over the system (typically just on/off).
I believe “sufficiently shorter than the delay” is just going to be Nyquist Shannon sampling theorem, once you’re sampling twice the frequency of the highest frequency dynamic in the system, your control system has all the information from the sensor and sampling more will not tell you anything else.
This is partly a terminology issue. By “controlling a variable” I mean “taking actions as necessary to keep that variable at some reference level.” So I say that the thermostat is controlling the temperature of the room (or if you want to split hairs, the temperature of the temperature sensor—suitably siting that sensor is an important part of a practical system). In the same sense, the body controls its core temperature, its blood oxygenation level, its posture[1], and many other things, and its actions to obtain those ends include sweating, breathing, changing muscle tensions, etc.
By the “output” or “action” of a control system I mean the actions it takes to keep the controlled variable at the reference. For the thermostat, this is turning the heat source on and off. It is not “controlling” (in the sense I defined) the rate of adding heat. The thermostat does not know how much heat is being delivered, and does not need to.
The resulting behaviour of the system is to keep the temperature of the room between two closely spaced levels: the temperature at which it turns the heat on, and the slightly higher temperature at which it turns the heat off. The rate at which the temperature goes up or down does not matter, provided the heat source is powerful enough to replenish all the energy leaking out of the walls, however cold it gets outside. If the heat source were replaced by one delivering twice as much power, the performance of the thermostat would be unchanged, except for being able to cope with more extreme cold weather.
The only delays in the thermostat itself are the time it takes for a mechanical switch to operate (milliseconds) and the time it takes for heat production to reach the sensor (minutes). These are so much faster than the changes in temperature due to the weather outside that it is most simply treated as operating in continuous time. There would be no practical benefit from sampling the temperature discretely and seeing how slow a sample rate you can get away with.
It sounded previously like you were making the strong claim that this setup can’t be applied to a closed control loop at all, even in e.g. the common (approximately universal?) case where we have a delay between the regulator’s action and it’s being able to measure that action’s effect. That’s mostly what I was responding to; the chaining that Alfred suggested in the sibling comment seems sensible enough to me.
It occurs to me that the household thermostat example is so non-demanding as to not be a poor intuition pump. I implicitly made the jump to thinking about a more demanding version of this without spelling that out. It’s always going to be a little silly trying to optimize an example that’s already intuitively good enough. Imagine for sake of argument a apparatus that needs tighter control such that there’s actually pressure to optimize beyond the simplest control algorithm.
Your examples of control systems all seem fine and accurate. I think we agree the tricky bit is picking the most sensible frame for mapping the real system to the diagram (assuming that’s roughly what you mean by terminology).
It seems like even with the improvements John Wentworth suggests there’s still some ambiguity in how to apply the result to a case where the regulator makes a time series of decisions, and you’re suggesting there’s some reason we can’t, or wouldn’t want to use discrete timesteps and chain/repeat the diagram.
At a little more length, I’m picturing the unrolling such that the current state is the sensor’s measurement time series through present, of which the regulator is certain. It’s merely uncertain about how its action—what fraction of the next interval to run the heat—will effect the measurement at future times. It’s probably easiest if we draw the diagram such that the time step is the delay between action and measured effect, and the regulator then sees the result of its action based on T1 at T3.
That seems pretty clearly to me to match the pattern this theorem requires, while still having a clear place to plug in whatever predictive model the regulator has. I bring up the sampling theorem as that is the bridge between the discrete samples we have and the continuous functions and differential equations you elsewhere say you want to use. Or stated a little more broadly, that theorem says we can freely move between continuous and discrete representations as needed, provided we sample frequently enough and the functions are well enough behaved to be amenable to calculus in the first place.
It sounded previously like you were making the strong claim that this setup can’t be applied to a closed control loop at all, even in e.g. the common (approximately universal?) case where we have a delay between the regulator’s action and it’s being able to measure that action’s effect.
I am making that claim. Closed loops have circular causal links between the (time-varying) variables. The SZR diagram that I originally objected to is acyclic, therefore it does not apply to closed loops.
Loop delays are beside the point. Sampling on that time scale is not required and may just degrade performance.
Imagine for sake of argument a apparatus that needs tighter control such that there’s actually pressure to optimize beyond the simplest control algorithm.
You are assuming that that tighter control demands the sort of more complicated algorithms that you are imagining, that predict how much heat to inject, based on a model of the whole environment, and so on.
Let’s look outward at the real world. All you need for precision temperature control is to replace the bang-bang control with a PID controller and a scheme for automatically tuning the PID parameters, and there you are. There is nothing in the manual for the ThermoClamp device to suggest a scheme of your suggested sort. In particular, like the room thermostat, the only thing it senses is the actual temperature. Nothing else. For this it uses a thermocouple, which is a continuous-time device, not sampled. There is also no sign of any model. I don’t know how this particular device tunes its PID parameters (probably a trade secret), but googling for how to auto-tune a PID has not turned up anything suggesting a model, only injecting test signals and adjusting the parameters to optimise observed performance—observed solely through measuring the controlled variable.
Everything is digital these days, but a modern automatic pilot is still sampling the sensors many times a second, and I’m sure that’s also true of the digital parts of the ThermoClamp. The time step is well below the characteristic timescales of the system being controlled. It has to be.
People talk about eliminating the cycles by unrolling. I believe this does not work. In causal graphs as generally understood, each of the variables is time-varying. In the unrolled version, each of the nodes represents the value of a single variable at a single point in time, so it’s a different sort of thing. Unrolling makes the number of nodes indefinitely large, so how are you going to mathematically talk about it? Only by taking advantage of its repetitive nature, and then you’re back to dealing with cyclic causal graphs while pretending you aren’t. “Tell me you’re using cyclic causal graphs without telling me you’re using cyclic causal graphs.”
Most centrally I think we’re seeing fundamentally different things with the causal graph. Or more to the point, I haven’t the slightest idea how one is supposed to do any useful reasoning with time varying nodes without somehow expanding it to consider how one node’s function and/or time series effects it’s leaf nodes (or another way, specifically what temporal relation the arrow represents). It also seems fairly inescapable to me that any way you consider that relation, an actual causal cycle where A causes B causes C causes A at the same instant looks very different than one where they indirectly effect each-other at some later time, to the point of needing different tools to analyze the two cases. The latter looks very much like the sort of thing solved with recursion or update loops in programs all the time. Alternately diff eq in the continuous case. The former looks like the sort of thing you need a solver to look for a valid solution for.
It’s fairly obvious why cycles of the first kind I describe would need different treatment—the graph would place constraints on valid solutions but not tell you how to find them. I’m not seeing how the second case is cyclic in the same sense and how you couldn’t just use induction arguments to extend to infinity.
AFAICT you and I aren’t disagreeing on anything about real control systems. It’s difficult to find a non-contrived example because so many control systems either aren’t that demanding or have a human in the loop. But this theorem is about optimal control systems, optimal in the formal computer science sense, so the fact that neither of us can come up with an example that isn’t solved by a PID control loop or similar is somewhat besides the point.
While PID controllers are applicable to many control problems and often perform satisfactorily without any improvements or only coarse tuning, they can perform poorly in some applications and do not in general provide optimal control.
The first example that comes to mind would be a position controller for a robot which uses only wheel odometry to estimate the current position. The robot doesn’t actually measure its position (the control outcome), but rather estimates it from wheel travel by assuming some model of traction.
In that situation the robot is controlling estimated position, not real position. If the estimate is always accurate then the distinction will not matter, but if not, and especially if it accumulates a drift, it will.
I believe you are looking for the distinction between closed loop and open loop controllers. IIUC this theorem only applies to open loop controllers. OP’s flowchart does not contain a feedback loop. For comparison, Richard Kennaway’s flowchart has a feedback loop between Z and R.
An example of an open loop controller is a dishwasher or laundry machine.
All animals are examples of closed loop controllers.
I consider “open loop control” almost a contradiction in terms. I mean, I don’t have anything to quarrel with in the Wikipedia article, but it’s not an effective way to actually control anything, unless there’s a closed-loop controller wrapped around it. (Often this takes the form of a human being inspecting and adjusting the equipment.) One might as well call the screws holding a piece of furniture together an open-loop controller.
Perhaps I’m off on this but wanted to just ask. How much of that term being a contradiction is driven by the lack of a good underlying model of something. I’ll use the house temperature example here.
We have a very good model (by assumption here) of energy transmission from the exterior to the interior. We have a controller (thermostat) that only measures external features, say outside temp and maybe light and a time duration for when and how long to run either the air conditioner or heating. With a good model (and probably a good initialization at installation) that regulator seems like it would do a good job of hitting the defined internal temp target without needing to monitor the internal temp.
That certainly fails the GRT on the grounds it’s a lot more complicated than it needs to be as measuring the internal temp and then activating the AC or heating is pretty simple and straightforward. But having a good model seems to fit well with the Shannon entropy aspects—the model itself tells us what is needed about the internal temp so monitoring/sampling that variable is not necessary.
Is the complicated regulation no an open loop control case? Or is it still something of a contradiction in terms in your view?
There’s the rub. Your model will have to not only know the external temperature, but also the conductivity of the walls, the power delivered or extracted by the heat/cold source, the people or pet animals in the room and how much heat they are generating, the temperature in the adjoining rooms, and all the other factors affecting the temperature that the designer may not have thought of.
All to avoid the cost of a thermometer which you’ll need anyway to calibrate the model. This is Heath Robinson, not practical engineering.
The ordinary room thermostat does not measure S. It measures Z. Its actions are determined by Z and the reference temperature set by the user (in your example assumed constant and omitted for simplicity). There is no causal arrow from S to R. Here is the true causal diagram:
The room thermostat is therefore outside the scope of the GRT. This is true of every control system that senses the variable it is controlling, i.e. every control system.
The diagram is a causal Bayes net which is a DAG so it can’t contain cycles. Your diagram contains a cycle between R and Z. The diagram I had in mind when writing the post was something like:
which is a thermostat over a single timestep.
If you wanted to have a feedback loop over multiple timesteps, you could conjoin several of these diagrams:
Each node along the top row is the temperature at successive times. Each node along the bottom row is the controller state at different times.
Indeed. So much the worse for causal Bayes nets. People have tried to extend Pearl-type causal analysis to cyclic causal graphs, but have not got very far. The limits of our tools are not the limits of the world.
I am not convinced by the idea of unrolling the graph in time. Outside of digital electronics, time is continuous [1]. There is no time step for the room thermostat. For more complicated systems, the causal relationships you see when you discretise time may vary depending on the chosen time step. Continuous time and differential equations are the right abstraction.
It would not be helpful to bring up speculations about the ultimately discrete nature of space-time on the Planck scale.
Oh, I see the misunderstanding! So, what you can do here is fix the issue by taking the limit as the time step goes to 0, or equivalently, saying
What we need to do here is remember that both
The unrolled loop still should not have the arrow between S and R, and should have the feedback, because the controller is sensing the outcome. Unrolled it should go: S1 → Z1, R1->Z1, Z1 → R2, R2 → Z2, S2-> Z2 …
How do you figure a thermostat directly measures what it’s controlling? It controls heat added/removed per unit time, typically just more/less/no change, and measures the resulting temperature at a single point on typically a minute+ delay due to the dynamics of the system (air and heat take time to diffuse, even with a blower). Any time step sufficiently shorter than that delay is going to work the same. The current measurement depends on what the thermostat did tens of seconds if not minutes previously.
There are times the continuous/discrete distinction is very important but this example isn’t one of them. As soon as you introduce a significant delay between cause and effect the time step model works (it may well be a dependence on multiple previous time-steps, but not the current one).
I don’t think this is an unusual example, we have a small number of sensors, we get data on a delay, and we’re actually trying to control e.g. the temperature in the whole house, holding a set point, minimizing variation between rooms and minimizing variation across time, with the smallest amount of control authority over the system (typically just on/off).
I believe “sufficiently shorter than the delay” is just going to be Nyquist Shannon sampling theorem, once you’re sampling twice the frequency of the highest frequency dynamic in the system, your control system has all the information from the sensor and sampling more will not tell you anything else.
This is partly a terminology issue. By “controlling a variable” I mean “taking actions as necessary to keep that variable at some reference level.” So I say that the thermostat is controlling the temperature of the room (or if you want to split hairs, the temperature of the temperature sensor—suitably siting that sensor is an important part of a practical system). In the same sense, the body controls its core temperature, its blood oxygenation level, its posture[1], and many other things, and its actions to obtain those ends include sweating, breathing, changing muscle tensions, etc.
By the “output” or “action” of a control system I mean the actions it takes to keep the controlled variable at the reference. For the thermostat, this is turning the heat source on and off. It is not “controlling” (in the sense I defined) the rate of adding heat. The thermostat does not know how much heat is being delivered, and does not need to.
The resulting behaviour of the system is to keep the temperature of the room between two closely spaced levels: the temperature at which it turns the heat on, and the slightly higher temperature at which it turns the heat off. The rate at which the temperature goes up or down does not matter, provided the heat source is powerful enough to replenish all the energy leaking out of the walls, however cold it gets outside. If the heat source were replaced by one delivering twice as much power, the performance of the thermostat would be unchanged, except for being able to cope with more extreme cold weather.
The only delays in the thermostat itself are the time it takes for a mechanical switch to operate (milliseconds) and the time it takes for heat production to reach the sensor (minutes). These are so much faster than the changes in temperature due to the weather outside that it is most simply treated as operating in continuous time. There would be no practical benefit from sampling the temperature discretely and seeing how slow a sample rate you can get away with.
How do we manage not to fall over? To walk and run? These are deep questions.
It sounded previously like you were making the strong claim that this setup can’t be applied to a closed control loop at all, even in e.g. the common (approximately universal?) case where we have a delay between the regulator’s action and it’s being able to measure that action’s effect. That’s mostly what I was responding to; the chaining that Alfred suggested in the sibling comment seems sensible enough to me.
It occurs to me that the household thermostat example is so non-demanding as to not be a poor intuition pump. I implicitly made the jump to thinking about a more demanding version of this without spelling that out. It’s always going to be a little silly trying to optimize an example that’s already intuitively good enough. Imagine for sake of argument a apparatus that needs tighter control such that there’s actually pressure to optimize beyond the simplest control algorithm.
Your examples of control systems all seem fine and accurate. I think we agree the tricky bit is picking the most sensible frame for mapping the real system to the diagram (assuming that’s roughly what you mean by terminology).
It seems like even with the improvements John Wentworth suggests there’s still some ambiguity in how to apply the result to a case where the regulator makes a time series of decisions, and you’re suggesting there’s some reason we can’t, or wouldn’t want to use discrete timesteps and chain/repeat the diagram.
At a little more length, I’m picturing the unrolling such that the current state is the sensor’s measurement time series through present, of which the regulator is certain. It’s merely uncertain about how its action—what fraction of the next interval to run the heat—will effect the measurement at future times. It’s probably easiest if we draw the diagram such that the time step is the delay between action and measured effect, and the regulator then sees the result of its action based on T1 at T3.
That seems pretty clearly to me to match the pattern this theorem requires, while still having a clear place to plug in whatever predictive model the regulator has. I bring up the sampling theorem as that is the bridge between the discrete samples we have and the continuous functions and differential equations you elsewhere say you want to use. Or stated a little more broadly, that theorem says we can freely move between continuous and discrete representations as needed, provided we sample frequently enough and the functions are well enough behaved to be amenable to calculus in the first place.
(I have been busy, hence the delay.)
I am making that claim. Closed loops have circular causal links between the (time-varying) variables. The SZR diagram that I originally objected to is acyclic, therefore it does not apply to closed loops.
Loop delays are beside the point. Sampling on that time scale is not required and may just degrade performance.
You are assuming that that tighter control demands the sort of more complicated algorithms that you are imagining, that predict how much heat to inject, based on a model of the whole environment, and so on.
Let’s look outward at the real world. All you need for precision temperature control is to replace the bang-bang control with a PID controller and a scheme for automatically tuning the PID parameters, and there you are. There is nothing in the manual for the ThermoClamp device to suggest a scheme of your suggested sort. In particular, like the room thermostat, the only thing it senses is the actual temperature. Nothing else. For this it uses a thermocouple, which is a continuous-time device, not sampled. There is also no sign of any model. I don’t know how this particular device tunes its PID parameters (probably a trade secret), but googling for how to auto-tune a PID has not turned up anything suggesting a model, only injecting test signals and adjusting the parameters to optimise observed performance—observed solely through measuring the controlled variable.
The early automatic pilots were analogue devices operating in continuous time.
Everything is digital these days, but a modern automatic pilot is still sampling the sensors many times a second, and I’m sure that’s also true of the digital parts of the ThermoClamp. The time step is well below the characteristic timescales of the system being controlled. It has to be.
People talk about eliminating the cycles by unrolling. I believe this does not work. In causal graphs as generally understood, each of the variables is time-varying. In the unrolled version, each of the nodes represents the value of a single variable at a single point in time, so it’s a different sort of thing. Unrolling makes the number of nodes indefinitely large, so how are you going to mathematically talk about it? Only by taking advantage of its repetitive nature, and then you’re back to dealing with cyclic causal graphs while pretending you aren’t. “Tell me you’re using cyclic causal graphs without telling me you’re using cyclic causal graphs.”
No worries, likewise.
Most centrally I think we’re seeing fundamentally different things with the causal graph. Or more to the point, I haven’t the slightest idea how one is supposed to do any useful reasoning with time varying nodes without somehow expanding it to consider how one node’s function and/or time series effects it’s leaf nodes (or another way, specifically what temporal relation the arrow represents). It also seems fairly inescapable to me that any way you consider that relation, an actual causal cycle where A causes B causes C causes A at the same instant looks very different than one where they indirectly effect each-other at some later time, to the point of needing different tools to analyze the two cases. The latter looks very much like the sort of thing solved with recursion or update loops in programs all the time. Alternately diff eq in the continuous case. The former looks like the sort of thing you need a solver to look for a valid solution for.
It’s fairly obvious why cycles of the first kind I describe would need different treatment—the graph would place constraints on valid solutions but not tell you how to find them. I’m not seeing how the second case is cyclic in the same sense and how you couldn’t just use induction arguments to extend to infinity.
AFAICT you and I aren’t disagreeing on anything about real control systems. It’s difficult to find a non-contrived example because so many control systems either aren’t that demanding or have a human in the loop. But this theorem is about optimal control systems, optimal in the formal computer science sense, so the fact that neither of us can come up with an example that isn’t solved by a PID control loop or similar is somewhat besides the point.
Although it’s true that the GRT is not applicable to a thermostat, there are a very wide class of controllers which do not measure their outcomes.
Can you give me some examples of what you have in mind?
The first example that comes to mind would be a position controller for a robot which uses only wheel odometry to estimate the current position. The robot doesn’t actually measure its position (the control outcome), but rather estimates it from wheel travel by assuming some model of traction.
In that situation the robot is controlling estimated position, not real position. If the estimate is always accurate then the distinction will not matter, but if not, and especially if it accumulates a drift, it will.
I believe you are looking for the distinction between closed loop and open loop controllers. IIUC this theorem only applies to open loop controllers. OP’s flowchart does not contain a feedback loop. For comparison, Richard Kennaway’s flowchart has a feedback loop between Z and R.
An example of an open loop controller is a dishwasher or laundry machine.
All animals are examples of closed loop controllers.
I consider “open loop control” almost a contradiction in terms. I mean, I don’t have anything to quarrel with in the Wikipedia article, but it’s not an effective way to actually control anything, unless there’s a closed-loop controller wrapped around it. (Often this takes the form of a human being inspecting and adjusting the equipment.) One might as well call the screws holding a piece of furniture together an open-loop controller.
Perhaps I’m off on this but wanted to just ask. How much of that term being a contradiction is driven by the lack of a good underlying model of something. I’ll use the house temperature example here.
We have a very good model (by assumption here) of energy transmission from the exterior to the interior. We have a controller (thermostat) that only measures external features, say outside temp and maybe light and a time duration for when and how long to run either the air conditioner or heating. With a good model (and probably a good initialization at installation) that regulator seems like it would do a good job of hitting the defined internal temp target without needing to monitor the internal temp.
That certainly fails the GRT on the grounds it’s a lot more complicated than it needs to be as measuring the internal temp and then activating the AC or heating is pretty simple and straightforward. But having a good model seems to fit well with the Shannon entropy aspects—the model itself tells us what is needed about the internal temp so monitoring/sampling that variable is not necessary.
Is the complicated regulation no an open loop control case? Or is it still something of a contradiction in terms in your view?
There’s the rub. Your model will have to not only know the external temperature, but also the conductivity of the walls, the power delivered or extracted by the heat/cold source, the people or pet animals in the room and how much heat they are generating, the temperature in the adjoining rooms, and all the other factors affecting the temperature that the designer may not have thought of.
All to avoid the cost of a thermometer which you’ll need anyway to calibrate the model. This is Heath Robinson, not practical engineering.