Suppose Alice’s daughter Beth gets cancer and slowly dies. After a long battle, numerous doctors that tell them Beth’s death is inevitable, and many nights in the hospital, Alice finally watches as Beth breathes her last. Then, Alice feels a stab of intense grief and goes into mourning for the next month.
Do you claim these negative emotions are a result of prediction error, and that Alice would feel zero grief if she only had an accurate understanding of the situation? Color me skeptical.
Another example: suppose Carl is tied to some train tracks and sees a train approaching. As the train gets closer, Carl feels an intense sense of fear, and anger against the person who tied him up. Do you claim this is also a prediction error? The bad thing that Carl is afraid of hasn’t actually happened yet (the train has yet to reach him); where exactly is the error located?
These are good questions. Thanks for the pushback :)
Do you claim these negative emotions are a result of prediction error, and that Alice would feel zero grief if she only had an accurate understanding of the situation?
Yes, but not in the way you think. “[Perfectly] accurate understanding of the situation”, such that there is no grief to experience, is an impossibly high standard. The implication of “If you’re sad you’re just a bad rationalist!” absolutely does not follow. It’s closer to the opposite, in that if you’re flinching from experiencing sadness (or other emotions) you’re resisting updating.
I give some explanation of how this relates to the process of grieving in a different comment downstream (ctrl-f “pet”), but there’s another aspect that I’d like to touch on here.
My Grandpa won life. The man was very successful in business, in marriage, in family. He lived into old age with a full mind, and as active as one can be in his later years. It’s really hard to expect more out of life than this, so when he finally croaked in his nineties… it’s kinda hard to expect more. I mean, yeah, it’d have been nice to have him for a few more years, and yeah, occasionally people live longer. Sure, it’d be nice for aging to have been solved. But overall it’s kinda like “That’s how it’s supposed to go. If only all life went so well”. At his funeral, there were a lot of people smiling and remembering him fondly.
In contrast, lose someone important who is in their twenties, and it’s devastating. There are going to be all sorts of ways in which you expected things to go differently, and updating your maps there (i.e. “grieving”) sucks. Alice’s death sucks not just because you would like more for her, but because you thought she would get more. And she didn’t. And that matters. These funerals are not so fun and full of smiles.
Carl is tied to some train tracks and sees a train approaching. As the train gets closer, Carl feels an intense sense of fear, and anger against the person who tied him up. Do you claim this is also a prediction error? The bad thing that Carl is afraid of hasn’t actually happened yet (the train has yet to reach him); where exactly is the error located?
Yes, most definitely.
The anger error is located in the mismatch between expecting the person who tied him up to have followed some norms which he clearly he wasn’t bound by, and the reality that he did not follow the norms. In that situation, I have a hard time imagining being angry because I can’t see why I’d ever put some expectations like that on someone who wasn’t bound by them. Even if it was my own brother I wouldn’t be angry because I’d be too shocked and confused to take my prediction error as his fault. Not “Fuck you for doing this to me” but “Fuck me for not recognizing this to be possible”.
The fear error is locating in the mismatch between expecting to be safe and the reality of not being safe. This one is utterly unreasonable to expect anyone to solve on the fly like that, but 1) when people resign themselves to their fate (the example given to me by the Jewish man who taught me this stuff was Jews in concentration camps), there is no more fear, and 2) when you can easily untie yourself minutes before the train gets there it’s not so scary anymore because you just get off the tracks.
It’s worth noting that these things can get pretty complicated, and fear doesn’t necessarily feel the way you’d expect when you actually find yourself in similar situations. For example, I have a friend whose rope harness came undone while rappelling, leaving him desperately clinging to the rope while trying to figure out how to safely get to the ground. Afterwards, he said that he was afraid until his harness fell apart. After that, he was just too busy figuring out what to do to feel afraid. Rising to the occasion often requires judiciously dropping prior expectations and coming up with new ones on the fly.
Emotions evolved as a way of influencing our behavior in useful directions. They correspond (approximately—this is evolution we’re talking about) to a prediction that there is some useful way of changing your behavior in response to a situation. Fear tells you take precautions, anger tells you to retaliate, contempt tells you to reconsider your alliance, etc. (Scott Alexander has a post on ACX theorizing that general happiness and sadness are a way of telling you to take more/fewer risks, but I can’t find it at the moment.)
I think your examples of fear disappearing when people give up hope of escape are explained at least as well by this hypothesis as by yours. Also your example of your friend who “was afraid until his harness fell apart”—that was the moment when “taking precautions” stopped being a useful action, but it seems pretty weird to conjecture that that was the moment when his prediction error disappeared (was he predicting a 100% chance of the harness breaking? or even >50%?)
On my model, examples of people giving up anger when they accept physical determinism strike me as understandable but mistaken. They are reasoning that some person could not have done otherwise, and thus give up on changing the person’s behavior, which causes them to stop feeling anger. But this is an error, because a system operating on completely deterministic rules can still be altered by outside forces—such as a pattern of other people retaliating in certain circumstances.
On my model, the correct reason to get angry at a murderer, but not to get angry at a storm, is that murderers can (sometimes) be deterred, and storms cannot. I think the person who stops feeling anger has performed an incomplete reduction that doesn’t add up to normality.
Notice that my model provides an explanation for why different negative emotions occur in different circumstances: They recommend different actions. As far as I can see, you have not offered an explanation for why some prediction errors cause fear, others anger, others disgust.
Your model also appears to require that we hypothesize that the prediction errors are coming from some inner part of a person that can’t be questioned directly and is also very stupid. We seemingly have to believe that an 8-year-old is scared of the dark because some inner part of them still hasn’t figured out that, yes, it gets dark every night, dipshit (even though the 8yo will profess belief in this, and has overwhelming experience of this). This seems implausible and unfalsifiable.
Emotions evolved as a way of influencing our behavior in useful directions. They correspond (approximately—this is evolution we’re talking about) to a prediction that there is some useful way of changing your behavior in response to a situation. Fear tells you take precautions, anger tells you to retaliate, contempt tells you to reconsider your alliance, etc.
This isn’t an alternative hypothesis. It’s another part of the same picture.
Notice how it’s a new prediction about how your behavior needs to be changed? That’s because you’re learning that the path you’re currently on was built on false presumptions. Get your predictions right the first time, and none of this is needed.
On my model, examples of people giving up anger when they accept physical determinism strike me as understandable but mistaken. They are reasoning that some person could not have done otherwise, and thus give up on changing the person’s behavior, which causes them to stop feeling anger. [...] I think the person who stops feeling anger has performed an incomplete reduction that doesn’t add up to normalit
Anger is a good example of this.
If you’re running around in the fantasy world of “We’re all just going to be nice to each other, because that’s what we should do, and therefore we should wish only good things on everyone”, then a murderer breaks things. Anger is an appropriate response here, because if you suppress anger (because of rationalizing about determinism or whatever) then you end up saying stupid things like “He couldn’t have done any differently! One dead is enough, we don’t need another to die!”.
But this is a stupid way to go through life in the first place, because it’s completely delusional. When I say that I wouldn’t be angry at someone who tied me to the tracks, that doesn’t mean I’m incapable of retaliation. I’ve never been murdered or tied to train tracks, but one time some friends and I were attacked by strangers who I correctly inferred were carrying knives and willing to use them—and I wasn’t angry at them. But, rather than lamenting “Sigh, I guess he was bound to do this” when fleeing didn’t work, I turned and threw the guy to the ground. While I was smashing his face with my elbows, I wasn’t feeling “GRR! I HATE YOU!”. I was laughing about how it really was as incredibly easy as you’d think, and how stupid he had to be to force a fight with a wrestler that was significantly larger than himself.
Anger is a flinch. If you touch a hot stove, sure, it makes sense to flinch away from the stove. Keeping your hand there—rationalizing “determinism” or otherwise—would be foolish.
But also, maybe you wouldn’t be in this situation, if you weren’t holding to some silly nonsense like “My allies would never betray me”, and instead thought things through and planned accordingly.
And perhaps even more importantly, is your flinch actually gonna work? They often don’t. You don’t want to end up the fool in impotent rage blubbering about how someone did something “wrong” when you in fact do not have the power to enforce the norm you’re attached to. You want to be the person who can see crystal clear what is going on, what their options are, and who doesn’t hesitate to take them when appropriate. Anger, like flinching in general, is best as a transient event when we get surprised, and until we can reorient.
Your model also appears to require that we hypothesize that the prediction errors are coming from some inner part of a person that can’t be questioned directly and is also very stupid.
Heh. This is where we’re going to differ big time. There’s a gigantic inferential chasm here so none of this will ring true, but nevertheless here is my stance:
It is our outer wrappings that are very stupid most of the time. Even when your inner part is stupid too, that’s generally still the fault of the outer wrapping getting in the way and not doing its job. Stupid outer layer notwithstanding, that inner layer can quite easily be questioned directly. And updated directly.
The way I came to this view is by learning hypnosis to better fix these “irrational inner parts”, peeling the onion a layer at a time to come up with methods that work in increasingly diverse circumstances, and eventually recognizing that the things that work most generally are actually what we do by default—until we freak out and make up stories about how we “can’t” and “need hypnosis to fix these non-directly accessible very stupid inner parts”. Turns out those stories aren’t true, and “hypnosis” is an engineered way of fighting delusion with delusion. The stories feel true due to confirmation bias, until you tug at the seams.
Notice how it’s a new prediction about how your behavior needs to be changed? That’s because you’re learning that the path you’re currently on was built on false presumptions. Get your predictions right the first time, and none of this is needed.
It seems to me that you should change your behavior as circumstances change, even if the changes are completely expected. When you step into deep water, you should start swimming; when you step out of the water, you should stop trying to swim and start walking again. This remains true even if the changes are 100% expected.
that inner layer can quite easily be questioned directly. And updated directly.
Do you mean to say that you have some empirical way of measuring these “prediction errors” that you’re referring to, separately from the emotions you claim they explain?
Got any data you can share?
If you use your technique on an 8-year-old who is scared of the dark at night, do you actually predict your technique would reveal that they have a prediction that it won’t get dark at night? Would your technique allow you to “directly update” the 8yo so that they stop being scared of the dark?
It seems to me that you should change your behavior as circumstances change, even if the changes are completely expected. When you step into deep water, you should start swimming;
Yes, your behavior at time t = 0 and time t = 1 ought to be different even if the changes between these times are entirely predicted. But at t = 0, your planned behavior for t = 1 will be swimming if you foresee the drop off. If you don’t see the drop off, you get that “Woah!” that tells you that you need to change your idea of what behavior is appropriate for t >=1.
I guess I should have said “Notice how your planned behavior has to change”.
Do you mean to say that you have some empirical way of measuring these “prediction errors” that you’re referring to, separately from the emotions you claim they explain?
Well, if you were to walk outside and get rained on, would you experience surprise? If you walked outside and didn’t get rained on, would you feel surprised? The answers here tells you what you’re predicting.
If you use your technique on an 8-year-old who is scared of the dark at night, do you actually predict your technique would reveal that they have a prediction that it won’t get dark at night? Would your technique allow you to “directly update” the 8yo so that they stop being scared of the dark?
No, I wouldn’t expect the 8-year-old to be doing “I expect it to not get dark”, but rather something more like “I expect to be able to see a lack of monsters at all times”—which obviously conflicts with the reality that they cannot when the lights are out.
The way I’d approach this depends on the specific context, but I generally would not want to directly update the kids beliefs in any simple sort of way. I take issue with the assumption that fear is a problem in the first place, and generally find that in any case remotely like this, direct overwriting of beliefs is a bad thing.
Got any data you can share?
I’m 13 posts into a big sequence laying out my thoughts on this, and it’s full of examples where I’ve achieved what might seem like unusual results from a “this stuff is unconscious and hard” perspective, but which aren’t nearly so impressive once you see behind the curtain.
The one I posted today, for example, shows how I was able to get both of my daughters to be unafraid of getting their shots when they were two years old (separate instances, not twins), and how the active ingredient was “not giving a shit if they’re afraid of their shots”.
If you want more direct proof that I’m talking about real things, the best example would be the transcript where I helped someone greatly reduce his suffering from chronic pain through forum PMs, following the basic idea of “Obviously pain isn’t a problem, but this guy sure seems to think it is, so how is he going wrong exactly?”. That one did eventually overwrite his felt experience of the pain being a bad thing (for the most part), but it wasn’t so quick and direct because like with the hypothetical scared 8 year old, a direct overwrite would have been bad.
If you want to learn more about “direct overwriting”, then that’s the section on attention, and I explain how I was able to tell my wife to constrict her blood vessels to stop bleeding in about thirty seconds, and why that isn’t nearly as an extraordinary claim as it might seem like it should be.
I should probably throw together a sequence page, but for now they’re all on my user profile.
Well, if you were to walk outside and get rained on, would you experience surprise? If you walked outside and didn’t get rained on, would you feel surprised? The answers here tells you what you’re predicting.
I feel like I have experienced a lot of negative emotions in my life that were not particularly correlated with a feeling of surprise. In fact, I can recall feeling anger about things where I literally wrote down a prediction that the thing would happen, before it happened.
Conversely, I can recall many pleasant surprises, which involved a lot of prediction error but no negative emotions.
So if this is what you are relying on to confirm your theory, it seems pretty disconfirmed by my life experience. And I’m reasonably certain that approximately everyone has similar observations from their own lives.
I thought this was understood, and the only way I was taking your theory even mildly seriously was on the assumption that you meant something different from ordinary surprise.
No, I wouldn’t expect the 8-year-old to be doing “I expect it to not get dark”, but rather something more like “I expect to be able to see a lack of monsters at all times”
I find it quite plausible they would have a preference for seeing a lack of monsters. I do not find it remotely plausible that they would have a prediction of continuously being able to see a lack of monsters. That is substantially more stupid than the already-very-stupid example of not expecting it to get dark.
Are you maybe trying to refer to our models of how the world “should” work, rather than our models of how it does work? I’m not sure exactly what I think “should” is, but I definitely don’t think it’s the same as a prediction about what actually will happen. But I could maybe believe that disagreements between “should” and “is” models play a role in explaining (some) negative emotions.
If you want more direct proof that I’m talking about real things, the best example would be the transcript where I helped someone greatly reduce his suffering from chronic pain through forum PMs
I am not searching through everything you’ve ever written to try to find something that matches a vague description.
I feel like we’ve been talking for quite a while, and you are making extraordinary claims, and you have not presented ANY noteworthy evidence favoring your model over my current one, and I am going to write you off very soon if I don’t see something persuasive. Please write or directly link some strong evidence.
I feel like I have experienced a lot of negative emotions in my life that were not particularly correlated with a feeling of surprise. In fact, I can recall feeling anger about things where I literally wrote down a prediction that the thing would happen, before it happened.
Ah, that’s what you’re getting at.
Okay, so for example, say you angrily tell your employee “I expect you to show up on time!”. Then, he doesn’t, and you’re not surprised. This shows that you (meta) expected your (object level) expectation of “You will show up on time!” to be false. You’re not surprised because you’re not learning anything, because you’ve chosen not to. Notice the hesitance to sigh and say “Well, I guess he is not going to show up on time”?
This stickiness comes from the desire to control things combined with a lack of sophisticated methods of control. When you accept “He is not going to show up on time”, you lose your ability to tell him “I expect you to show up on time!” and with it your ability too put pressure on him to be punctual. Your setpoint that you control to is your expectation, so if you update your expectation then you lose your ability to (crudely) attempt to control the person’s behavior. Once you learn more sophisticated methods of control, the anger no longer serves a purpose so you’re free to update your expectations to match reality. E.g. “I don’t know if you’re going to show up on time, but I do know that if you don’t, you will be fired! No hard feelings either way, have a nice day :)”
This is a really tricky equivalence to wrap ones mind around, and it took me years to really understand even after I could see that there was something there. I explain this more in my post expectations=intentions=setpoint, and give examples of how more sophisticated attempts to control cede immediate reality and attempt to control towards trajectories instead—to concretely better results.
Conversely, I can recall many pleasant surprises, which involved a lot of prediction error but no negative emotions.
Yeah, I think positive emotions generally require prediction errors too, though I’m less solid on this one. People are generally more willing to update on pleasant surprises so that prediction error discomfort is less likely to persist enough to be notable, though it’s worth noting that this isn’t always the case. Imposter syndrome is an example where people get quite uncomfortable because of this refusal.
The prediction error is not the same as negative emotions. Prediction error is the suffering that happens while you refuse to update, while negative emotions like sadness come while you update. You still have to have erred in order to have something sad to learn, but it’s not the same thing.
Now that I say it, I realize I had the opportunity to clarify earlier, because I did notice that this was a point of confusion, and I chose not to take the opportunity to. I think I see why I did, but I shouldn’t have, and I apologize.
So if this is what you are relying on to confirm your theory, it seems pretty disconfirmed by my life experience. And I’m reasonably certain that approximately everyone has similar observations from their own lives.
Again, giant chasm of inferential distance. You’re not passing the ITT yet, and until you do it’s going to be real tough for you to “test” anything I’m saying, because you will always be testing a misinterpretation. It’s obviously reasonable to be suspicious of such claims as attempts to hide from falsifiability, but at the same time sometimes that’s just how things are—and assuming either way is a poor way of finding truth.
To distinguish, you want to look for signs of cognitive dissonance, not merely things that you disagree with. Because if you conclude that you’re right on the surface level because the other person gets something wrong two levels deep… and your judgment of whether they’re wrong is your perspective, which they disagree with… then you’ve just given up on ever learning anything when there’s a disagreement three or more levels deep. If you wait to see signs that the person is being forced to choose between changing their own mind or ignoring data, then you have a much more solid base.
That, and look for concrete predictions that both sides can agree on. For example, you took the stance that anger was appropriate because without it you become susceptible to murderers and don’t retaliate—but once I pointed out the alternative, I don’t think you doubt that I was actually able to fight back without anger? Or is that genuinely hard to believe for you?
I find it quite plausible they would have a preference for seeing a lack of monsters. I do not find it remotely plausible that they would have a prediction of continuously being able to see a lack of monsters. That is substantially more stupid than the already-very-stupid example of not expecting it to get dark.
Hey, kids are stupid. Adults too. Sometimes people even keep expecting people to not piss them off, even when they know that the person will piss them off :p
Jokes aside, this is still the “expectations=intentions” thing. We try to not see our expectations as predictions when we’re using them as intents, but they function as predictions nonetheless.
Are you maybe trying to refer to our models of how the world “should” work, rather than our models of how it does work? I’m not sure exactly what I think “should” is, but I definitely don’t think it’s the same as a prediction about what actually will happen. But I could maybe believe that disagreements between “should” and “is” models play a role in explaining (some) negative emotions.
“Should” is used as an attempt to disconnect ourselves from updating on what will happen in order to try to make something happen—because we recognize that it will likely fail and want to try anyway. If I say “You will show up on time” as if it’s a matter of fact, that’s either powerful… or laughable. And if I sense that I don’t have the authority to pull that off, I’m incentivized to back off to “You SHOULD show up on time” so that I don’t have to accept “I guess you won’t show up on time, huh?” when you don’t. I can always say “Okay maybe he won’t BUT HE SHOULD” and immediately negate the uncomfortable reality.
So “Yes, I’m talking about our models of how the world should work”, and also that is necessarily the same as our models of how the world does work—even if we also have meta models which identify the predictable errors in our object level models and try to contain them.
Maybe that part could use more emphasis. Of course we have meta models that contradict these obviously wrong object level models. We know that we’re probably wrong, but on the object level that doesn’t make us any less wrong until we actually do the update.
I am not searching through everything you’ve ever written to try to find something that matches a vague description.
That’s fine, no pressure to do anything of course. For what it’s worth though, it’s very clearly labeled. There’s no way you wouldn’t recognize at a glance.
I feel like we’ve been talking for quite a while, and you are making extraordinary claims, and you have not presented ANY noteworthy evidence favoring your model over my current one,
I don’t think that’s fair. For one, your model said you need anger in order to retaliate, and I gave an example of how I didn’t need anger in order to retaliate. I think the fact that I don’t always struggle with predictable anger while simultaneously not experiencing your predicted downsides is clear evidence, do you not?
Of course, this isn’t strong evidence that I’m right about anything else, but it’s absolutely fatal to the idea that your model accurately depicts the realm of possibility. If your model gets one thing this wrong, this unexpectedly, how can you trust it to tell you what else to view as “extraordinary”?
and I am going to write you off very soon if I don’t see something persuasive. Please write or directly link some strong evidence.
You’re welcome to read my posts, or not. They’re quite long and I don’t expect you to read them, but they’re there if you want a better understanding of what I’m talking about.
Either way, I’m happy to continue because I can see that you’re engaging in good faith even though you’re skeptical (and maybe a bit frustrated?), and I appreciate the push back. At the end of the day, neither my ability to retaliate without anger, nor my ability to help kids overcome fear by understanding their predictions, hinge on you believing in it.
At the same time, I’m curious if you’ve thought about how it looks from my perspective. You’ve written intelligent and thoughtful responses which I appreciate, but are you under the impression that anything you’ve written provides counter-evidence? Do you picture me thinking “Yes, that’s what I’m saying” before you argue against what you think I’m saying?
I don’t think that’s fair. For one, your model said you need anger in order to retaliate, and I gave an example of how I didn’t need anger in order to retaliate.
I didn’t respond to this because I didn’t see it as posing any difficulty for my model, and didn’t realize that you did.
I don’t think you need anger in order to retaliate. I think anger means that the part of you that generates emotions (roughly, Kahneman’s system 1) wants to retaliate. Your system 2 can disagree with your system 1 and retaliate when you’re not angry.
Also, your story didn’t sound to me like you were actually retaliating. It sounded to me like you were defending yourself, i.e. taking actions that reduced the other guy’s capability of harming you. Retaliation (on my model) is when you harm someone else in an effort to change their decisions (not their capabilities), or the decisions of observers.
So I’m quite willing to believe the story happened as you described it, but this was 2 steps removed from posing any problem to my model, and you didn’t previously explain how you believed it posed a problem.
I also note that you said “for one” (in the quote above) but then there was no number two in your list.
If you wait to see signs that the person is being forced to choose between changing their own mind or ignoring data, then you have a much more solid base.
I do see a bunch of signs of that, actually:
I claimed that your example of your friend being afraid until their harness broke seems to be better explained by my model than yours, because that would be an obvious time for the recommended action to change but a really weird time for his prediction error to disappear. You did not respond to this point.
I claimed that my model has an explanation for how different negative emotions are different and why you experience different ones in different situations, and your model seemingly does not, and this makes my model better. You did not respond to this point.
I asked you if you had a way of measuring whatever you mean by “prediction error”, so that we could check how well the measurements fit your model. You told me to use my own feelings of surprise. When I pointed out that doesn’t mach your model, you said that you meant something different, but didn’t clarify what you meant, and did not provide a new answer to the earlier question about how you measure “prediction error”. This looks like you saying whatever deflects the current point without keeping track of how the current point is related to previous points.
Note that I don’t actually need to understand what you mean in order for the measurement to be interesting. You could hand me a black box and say “this measures the thing I’m talking about” and if the black box produces measurements that correlate with your predictions that would be interesting even if I have no clue how the black box works (as long as I don’t see an uninteresting way of deriving your predictions from its inputs). But you haven’t done this, either.
I gave an example where I made an explicit prediction, and then was angry when it came true. You responded by ignoring my example and substituting your own hypothetical example where I made an explicit prediction and then was angry when it was falsified. This looks like you shying away from examples that are hard for your theory to explain and instead rehearsing examples that are easier.
You have claimed that there’s evidence in your other writing, but have refused to prioritize it so that I can find your best evidence as quickly as possible. This looks like an attempt to dissuade me from checking your claims by maximizing the burden of effort placed on me. In a cooperative effort of truth-seeking, you ought to be the one performing the prioritization of your writing because you have a massive advantage in doing so.
Many of your responses seem like you are using my points to launch off on a tangent, rather than addressing my point head-on.
So “Yes, I’m talking about our models of how the world should work”, and also that is necessarily the same as our models of how the world does work—even if we also have meta models which identify the predictable errors in our object level models and try to contain them.
This seems like it’s just a simple direct contradiction. You’re saying that model X and model Y are literally the same thing, but also that we keep track of the differences between them. There couldn’t be any differences to track if they were actually the same thing.
I also note that you claimed these are “necessarily” the same, but provided no reasoning or evidence to back that up; it’s just a flat assertion.
At the same time, I’m curious if you’ve thought about how it looks from my perspective. You’ve written intelligent and thoughtful responses which I appreciate, but are you under the impression that anything you’ve written provides counter-evidence? Do you picture me thinking “Yes, that’s what I’m saying” before you argue against what you think I’m saying?
There are some parts of your model that I think I probably roughly understand, such as the fact that you think there’s some model inside a person making predictions (but it’s not the same as the predictions they profess in conversation) and that errors in these predictions are a necessary precondition to feeling negative emotions. I think I can describe these parts in a way you would endorse.
There are some parts of your model that I think I probably don’t understand, like where is that model actually located and how does it work.
There are some parts of your model that I think are incoherent bullshit, like where you think “should” and “is” models are the same thing but also we have a meta-model that tracks the differences between them, or where you think telling me to pay attention to my own feelings of surprise makes any sense as a response to my request for measurements.
I don’t think I’ve written anything that directly falsifies your model as a whole—which I think is mostly because you haven’t made it legible enough.
But I do think I’ve pointed out:
several ways in which my model wins Bayes points against yours
several ways that your model creates more friction than mine with common-sensical beliefs across other domains
several ways in which your own explanations of your model are contradictory or otherwise deficient
that there is an absence of support on your side of the discussion
I don’t think I require a better understanding of your model than I currently have in order for these points to be justified.
You’re extending yourself an awful lot of charity here.
For example, you accuse me of failing to respond to some of your points, and claim that this is evidence of cognitive dissonance, yet you begin this comment with:
I didn’t respond to this because I didn’t see it as posing any difficulty for my model, and didn’t realize that you did.
Are you really unable to anticipate that this is very close to what I would have said, if you had asked me why I didn’t respond to those things? The only reason that wouldn’t be my exact answer is that I’d first point out that I did respond to those things, by pointing out that your arguments were based on a misunderstanding of my model! This doesn’t seem like a hard one to get right, if you were extending half the charity to me that you extend yourself, you know? (should I be angry with you for this, by the way?)
As to your claim that it doesn’t pose difficulty to your model, and attempts to relocate goal posts, here are your exact words:
I think the person who stops feeling anger has performed an incomplete reduction that doesn’t add up to normality.
This is wrong. It is completely normal to not feel anger, and retaliate, when you have accurate models instead of clinging to inaccurate models, and I gave an example of this. Your attempt to pick the nit between “incapacitation” vs “dissuasion” is very suspect as well, but also irrelevant because dissuasion was also a goal (and effect) of my retaliation that night. I could give other examples too, which are even more clearly dissuasion not incapacitation, but I think the point is pretty clear.
And no, even with the relocated goalposts your explanation fails. That was a system 1 decision, and there’s no time for thinking slow when you’re in the midst of something like that.
You have claimed that there’s evidence in your other writing, but have refused to prioritize it so that I can find your best evidence as quickly as possible.
No, I made it very clear. If you have a fraction of the interest it would take to read the post and digest the contents, you would spend the ten seconds needed to pull up the post. This is not a serious objection.
Again, it’s totally understandable if you don’t want to take the time to read it. It’s a serious time and effort investment to sit down and not only read but make sense of the contents, so if your response were to be “Hey man, I got a job and a life, and I can’t afford to spend the time especially given that I can’t trust it’ll change my mind”, that would be completely reasonable.
But to act like “Nope, it doesn’t count because you can’t expect me to take 10 seconds to find it, and therefore must be trying to hide it” is.… well, can you see how that might come across?
>So “Yes, I’m talking about our models of how the world should work”, and also that is necessarily the same as our models of how the world does work—even if we also have meta models which identify the predictable errors in our object level models and try to contain them.
This seems like it’s just a simple direct contradiction. You’re saying that model X and model Y are literally the same thing, but also that we keep track of the differences between them. There couldn’t be any differences to track if they were actually the same thing.
So if I tell you that the bottle of distilled water with “Drinking water” scribbled over the label contains the same thing as the bottle of distilled water that has “coolant” scribbled on it… and that the difference is only in the label… would you understand that? Would that register to you as a coherent possibility?
I’m sorry, but I’m having a hard time understanding which part of this is weird to you. Are you really claiming that you can’t see how to make sense of this?
>At the same time, I’m curious if you’ve thought about how it looks from my perspective.
There are some parts of your model that I think I probably roughly understand, [...] But I do think I’ve pointed out:
You’re missing the point of my question. Of course you think you’ve pointed that stuff out. I’m not asking if you believe you’re justified in your own beliefs.
There are a lot of symmetries here. You said some thing’s that [you claim] I didn’t respond to. I said some things which [I claim] you didn’t respond to. Some of the things I say strike you as either missing the point or not directly responding to what you say. A lot of the things that you’ve said strike me in the same way. Some of the my responses [you claim] look like cognitive dissonance to you. Some of your responses [I claim] look that way to me. I’m sure you think it’s different because your side really is right, and my side really is wrong. And of course, I feel the same way. This is all completely normal for disagreements that run more than a step or two deep.
But then you go on to act like you don’t notice the symmetry, as if your own perspective objectively validates your own side. You start to posture stuff like “You haven’t posted any evidence [that I recognize]” and “I’m gonna write you off, if you don’t persuade me”, with no hint to the possibility that there’s another side to this coin.
The question is, do you see how silly this looks, from my perspective? Do you see how much this looks like you’re missing the self awareness that is necessary in order to have a hope of noticing when you’re inhabiting a mistaken worldview, which pats itself on the back prematurely?
Because if you do, then perhaps we can laugh about our situation together, and go about figuring out how to break this asymmetry. But if you don’t, or if you try to insist “No, but my perspective really is better supported [according to me]”, the symmetry is already broken.
Are you really unable to anticipate that this is very close to what I would have said, if you had asked me why I didn’t respond to those things? The only reason that wouldn’t be my exact answer is that I’d first point out that I did respond to those things, by pointing out that your arguments were based on a misunderstanding of my model! This doesn’t seem like a hard one to get right, if you were extending half the charity to me that you extend yourself, you know? (should I be angry with you for this, by the way?)
You complain that I failed to anticipate that you would give the same response as me, but then immediately give a diametrically opposed response! I agreed that I didn’t respond to the example you highlighted, and said this was because I didn’t pick up on your implied argument. You claim that you did respond to the examples I highlighted. The accusations are symmetrical, but the defenses are very much not.
I did notice that the accusations were symmetrical, and because of that I very carefully checked (before posting) whether the excuse I was giving myself could also be extended to you, and I concluded definitively that it couldn’t. My examples made direct explicit comparisons between my model and (my model of) your model, and pointed out concrete ways that the output of my model was better; it seems hugely implausible you failed to understand that I was claiming to score Bayes points against your model. Your example did not mention my model at all! (It contrasts two background assumptions, where humans are either always nice or not, and examines how your model, and only your model, interacts with each of those assumptions. I note that “humans are always nice” is not a position that anyone in this thread has ever defended, to my knowledge.)
And yes, I did also consider the meta-level possibility that my attempt to distinguish between what was said explicitly and what wasn’t is so biased as to make its results useless. I have a small but non-zero probability for that. But even if that’s true, that doesn’t seem like a reason to continue the argument; it seems like proof that I’m so hopeless that I should just cut my losses.
I considered including a note in my previous reply explaining that I’d checked if you could use my excuse and found you couldn’t, but I was concerned that would feel like rubbing it in, and the fact that you can’t use my excuse isn’t actually important unless you try to use it, and I guessed that you wouldn’t try. (Whether that guess was correct is still a bit unclear to me—you offer an explanation that seems directly contradictory to my excuse, but you also assert that you’re saying the same thing as me.)
If you are saying that I should have guessed the exact defense you would give, even if it was different from mine, then I don’t see how I was supposed to guess that.
If you are saying that I should have guessed you would offer some defense, even if I didn’t know the details, then I considered that moderately likely but I don’t know what you think I should have done about it.
If I had guessed that you would offer some defense that I would accept then I could have updated to the position I expected to hold in the future, but I did not guess that you’d have a defense I would accept; and, in fact, you don’t have one. Which brings us to...
(re-quoted for ease of reference)
I did respond to those things, by pointing out that your arguments were based on a misunderstanding of my model!
I have carefully re-read the entire reply that you made after the comment containing the two examples I accused you of failing to respond to.
Those two examples are not mentioned anywhere in it. Nor is there a general statement about “my examples” as a group. It has 3 distinct passages, each of which seems to be a narrow reply to a specific thing that I said, and none of which involve these 2 examples.
Nor does it include a claim that I’ve misapplied your model, either generally or related to those particular examples. It does include a claim that I’ve misunderstood one specific part of your model that was completely irrelevant to those two examples (you deny my claim that the relevant predictions are coming from a part of the person that can’t be interrogated, after flagging that you don’t expect me to follow that passage due to inferential distance).
Your later replies did make general claims about me not understanding your model several times. I could make up a story where you ignored these two examples temporarily and then later tried to address them (without referencing them or saying that that was what you were doing), but that story seems neither reasonable nor likely.
Possibly you meant to write something about them, but it got lost in an editing pass?
Or (more worryingly) perhaps you responded to my claim that you had ignored them not by trying to find actions you took specifically in response to those examples, but instead by searching your memory of everything you’ve said for things that could be interpreted as a reply, and then reported what you found without checking it?
In any case: You did not make the response you claimed that you made, in any way that I can detect.
Communication is tricky!
Sometimes both parties do something that could have worked, if the other party had done something different, but they didn’t work together, and so the problem can potentially be addressed by either party. Other times, there’s one side that could do something to prevent the problem, but the other side basically can’t do anything on their own. Sometimes fixing the issue requires a coordinated solution with actions from both parties. And in some sad situations, it’s not clear the issue can be fixed at all.
It seems to me that these two incidents both fall clearly into the category of “fixable from your side only”. Let’s recap:
(1) When you talked about your no-anger fight, you had an argument against my model, but you didn’t state it explicitly; you relied on me to infer it. That inference turned out to be intractable, because you had a misunderstanding about my position that I was unaware of. (You hadn’t mentioned it, I had no model that had flagged that specific misunderstanding as being especially likely, and searching over all possible misunderstandings is infeasible.)
There’s an obvious, simple, easy, direct fix from your side: State your arguments explicitly. Or at least be explicit that you’re making an argument, and you expect credit. (I mistook this passage as descriptive, not persuasive.)
I see no good options from my side. I couldn’t address it directly because I didn’t know what you’d tried to do. Maybe I could have originally explained my position in a way that avoided your misunderstanding, but it’s not obvious what strategy would have accomplished that. I could have challenged your general absence of evidence sooner—I was thinking it earlier, but I deferred that option because it risked degrading the conversation, and it’s not clear to me that was a bad call. (Even if I had said it immediately, that would presumably just accelerate what actually happened.)
If you have an actionable suggestion for how I could have unilaterally prevented this problem, please share.
(2) In the two examples I complained you didn’t respond to, you allege that you did respond, but I didn’t notice and still can’t find any such response.
My best guess at the solution here is “you need to actually write it, instead of just imagining that you wrote it.” The difficulty of implementing that could range from easy to very hard, depending on the actual sequence of events that lead to this outcome. But whatever the difficulty, it’s hard to imagine it could be easier to implement from my side than yours—you have a whole lot of relevant access to your writing process that I lack.
Even assuming this is a problem with me not recognizing it rather than it not existing, there are still obvious things you could do on your end to improve the odds (signposting, organization, being more explicit, quoting/linking the response when later discussing it). Conversely, I don’t see what strategy I could have used other than “read more carefully,” but I already carefully re-read the entire reply specifically looking for it, and still can’t find it.
I understand it’s possible to be in a situation where both sides have equal quality but both perceive themselves as better. But it’s also possible to be in a situation where one side is actually better and the other side falsely claims it’s symmetrical. If I allowed a mere assertion of symmetry from the other guy to stop me from ever believing the second option, I’d get severely exploited. The only way I have a chance at avoiding both errors is by carefully examining the actual circumstances and weighing the evidence case-by-case.
My best judgment here is that the evidence weighs pretty heavily towards the problems being fixable from your side and not fixable from my side. This seems very asymmetrical to me. I think I’ve been as careful as I reasonably could have been, and have invested a frankly unreasonable amount of time into triple-checking this.
Before I respond to your other points, let me pause and ask if I have convinced you that our situation is actually pretty asymmetrical, at least in regards to these examples? If not, I’m disinclined to invest more time.
Before I respond to your other points, let me pause and ask if I have convinced you that our situation is actually pretty asymmetrical, at least in regards to these examples? If not, I’m disinclined to invest more time.
Oh, the situation is definitely asymmetrical. In more ways than you realize.
However, the important part of my comment was this:
The question is, do you see how silly this looks, from my perspective? Do you see how much this looks like you’re missing the self awareness that is necessary in order to have a hope of noticing when you’re inhabiting a mistaken worldview, which pats itself on the back prematurely?
Because if you do, then perhaps we can laugh about our situation together, and go about figuring out how to break this asymmetry. But if you don’t, or if you try to insist “No, but my perspective really is better supported [according to me]”, the symmetry is already broken.
If you can’t say “Shoot, I didn’t realize that”, or “Heh, yeah I see how it definitely looks more symmetrical than I was giving credit for (even though we both know there are important dissymmetries, and disagree on what they are)”, and instead are going to spend a lot of words insisting “No, but my perspective really is better supported [according to me]”… after I just did you the favor of highlighting how revealing that would be… then again, the symmetry is already broken in the way that shows which one of us is blind to our limitations.
There’s another asymmetry though, which has eluded you:
Despite threatening to write me off, you still take me seriously enough to write a long comment trying to convince me that you’re right, and expect me to engage with it. Since you failed to answer the part that matters, I can’t even take you seriously enough to read it. Ironically, this would have been predictable to you if not for your stance on prediction errors, Lol.
Also, with a prediction error like that, you’re probably not having as much fun as I am, which is a shame. I’m genuinely sorry it turned out the way it did, as I was hoping we’d get somewhere interesting with this. I hope you can resolve your error before it eats at you too much, and that you can keep a sense of humor about things :)
We can be, if you want. And I certainly wouldn’t blame you for wanting to bail after the way I teased you in the last comment.
I do want to emphasize that I am sincere in telling you that I hope it doesn’t eat at you too much, and that I hoped for the conversation to get somewhere interesting.
If you turn out to be a remarkably good sport about the teasing, and want to show me that you can represent how you were coming off to me, I’m still open to that conversation. And it would be a lot more respectful, because it would mean addressing the reason I couldn’t take your previous comment seriously.
No expectations, of course. Sincere best wishes either way, and I hope you forgive me for the tease.
Suppose Alice’s daughter Beth gets cancer and slowly dies. After a long battle, numerous doctors that tell them Beth’s death is inevitable, and many nights in the hospital, Alice finally watches as Beth breathes her last. Then, Alice feels a stab of intense grief and goes into mourning for the next month.
Do you claim these negative emotions are a result of prediction error, and that Alice would feel zero grief if she only had an accurate understanding of the situation? Color me skeptical.
Another example: suppose Carl is tied to some train tracks and sees a train approaching. As the train gets closer, Carl feels an intense sense of fear, and anger against the person who tied him up. Do you claim this is also a prediction error? The bad thing that Carl is afraid of hasn’t actually happened yet (the train has yet to reach him); where exactly is the error located?
These are good questions. Thanks for the pushback :)
Yes, but not in the way you think. “[Perfectly] accurate understanding of the situation”, such that there is no grief to experience, is an impossibly high standard. The implication of “If you’re sad you’re just a bad rationalist!” absolutely does not follow. It’s closer to the opposite, in that if you’re flinching from experiencing sadness (or other emotions) you’re resisting updating.
I give some explanation of how this relates to the process of grieving in a different comment downstream (ctrl-f “pet”), but there’s another aspect that I’d like to touch on here.
My Grandpa won life. The man was very successful in business, in marriage, in family. He lived into old age with a full mind, and as active as one can be in his later years. It’s really hard to expect more out of life than this, so when he finally croaked in his nineties… it’s kinda hard to expect more. I mean, yeah, it’d have been nice to have him for a few more years, and yeah, occasionally people live longer. Sure, it’d be nice for aging to have been solved. But overall it’s kinda like “That’s how it’s supposed to go. If only all life went so well”. At his funeral, there were a lot of people smiling and remembering him fondly.
In contrast, lose someone important who is in their twenties, and it’s devastating. There are going to be all sorts of ways in which you expected things to go differently, and updating your maps there (i.e. “grieving”) sucks. Alice’s death sucks not just because you would like more for her, but because you thought she would get more. And she didn’t. And that matters. These funerals are not so fun and full of smiles.
Yes, most definitely.
The anger error is located in the mismatch between expecting the person who tied him up to have followed some norms which he clearly he wasn’t bound by, and the reality that he did not follow the norms. In that situation, I have a hard time imagining being angry because I can’t see why I’d ever put some expectations like that on someone who wasn’t bound by them. Even if it was my own brother I wouldn’t be angry because I’d be too shocked and confused to take my prediction error as his fault. Not “Fuck you for doing this to me” but “Fuck me for not recognizing this to be possible”.
The fear error is locating in the mismatch between expecting to be safe and the reality of not being safe. This one is utterly unreasonable to expect anyone to solve on the fly like that, but 1) when people resign themselves to their fate (the example given to me by the Jewish man who taught me this stuff was Jews in concentration camps), there is no more fear, and 2) when you can easily untie yourself minutes before the train gets there it’s not so scary anymore because you just get off the tracks.
It’s worth noting that these things can get pretty complicated, and fear doesn’t necessarily feel the way you’d expect when you actually find yourself in similar situations. For example, I have a friend whose rope harness came undone while rappelling, leaving him desperately clinging to the rope while trying to figure out how to safely get to the ground. Afterwards, he said that he was afraid until his harness fell apart. After that, he was just too busy figuring out what to do to feel afraid. Rising to the occasion often requires judiciously dropping prior expectations and coming up with new ones on the fly.
Let me propose an alternate hypothesis:
Emotions evolved as a way of influencing our behavior in useful directions. They correspond (approximately—this is evolution we’re talking about) to a prediction that there is some useful way of changing your behavior in response to a situation. Fear tells you take precautions, anger tells you to retaliate, contempt tells you to reconsider your alliance, etc. (Scott Alexander has a post on ACX theorizing that general happiness and sadness are a way of telling you to take more/fewer risks, but I can’t find it at the moment.)
I think your examples of fear disappearing when people give up hope of escape are explained at least as well by this hypothesis as by yours. Also your example of your friend who “was afraid until his harness fell apart”—that was the moment when “taking precautions” stopped being a useful action, but it seems pretty weird to conjecture that that was the moment when his prediction error disappeared (was he predicting a 100% chance of the harness breaking? or even >50%?)
On my model, examples of people giving up anger when they accept physical determinism strike me as understandable but mistaken. They are reasoning that some person could not have done otherwise, and thus give up on changing the person’s behavior, which causes them to stop feeling anger. But this is an error, because a system operating on completely deterministic rules can still be altered by outside forces—such as a pattern of other people retaliating in certain circumstances.
On my model, the correct reason to get angry at a murderer, but not to get angry at a storm, is that murderers can (sometimes) be deterred, and storms cannot. I think the person who stops feeling anger has performed an incomplete reduction that doesn’t add up to normality.
Notice that my model provides an explanation for why different negative emotions occur in different circumstances: They recommend different actions. As far as I can see, you have not offered an explanation for why some prediction errors cause fear, others anger, others disgust.
Your model also appears to require that we hypothesize that the prediction errors are coming from some inner part of a person that can’t be questioned directly and is also very stupid. We seemingly have to believe that an 8-year-old is scared of the dark because some inner part of them still hasn’t figured out that, yes, it gets dark every night, dipshit (even though the 8yo will profess belief in this, and has overwhelming experience of this). This seems implausible and unfalsifiable.
This isn’t an alternative hypothesis. It’s another part of the same picture.
Notice how it’s a new prediction about how your behavior needs to be changed? That’s because you’re learning that the path you’re currently on was built on false presumptions. Get your predictions right the first time, and none of this is needed.
Anger is a good example of this.
If you’re running around in the fantasy world of “We’re all just going to be nice to each other, because that’s what we should do, and therefore we should wish only good things on everyone”, then a murderer breaks things. Anger is an appropriate response here, because if you suppress anger (because of rationalizing about determinism or whatever) then you end up saying stupid things like “He couldn’t have done any differently! One dead is enough, we don’t need another to die!”.
But this is a stupid way to go through life in the first place, because it’s completely delusional. When I say that I wouldn’t be angry at someone who tied me to the tracks, that doesn’t mean I’m incapable of retaliation. I’ve never been murdered or tied to train tracks, but one time some friends and I were attacked by strangers who I correctly inferred were carrying knives and willing to use them—and I wasn’t angry at them. But, rather than lamenting “Sigh, I guess he was bound to do this” when fleeing didn’t work, I turned and threw the guy to the ground. While I was smashing his face with my elbows, I wasn’t feeling “GRR! I HATE YOU!”. I was laughing about how it really was as incredibly easy as you’d think, and how stupid he had to be to force a fight with a wrestler that was significantly larger than himself.
Anger is a flinch. If you touch a hot stove, sure, it makes sense to flinch away from the stove. Keeping your hand there—rationalizing “determinism” or otherwise—would be foolish.
But also, maybe you wouldn’t be in this situation, if you weren’t holding to some silly nonsense like “My allies would never betray me”, and instead thought things through and planned accordingly.
And perhaps even more importantly, is your flinch actually gonna work? They often don’t. You don’t want to end up the fool in impotent rage blubbering about how someone did something “wrong” when you in fact do not have the power to enforce the norm you’re attached to. You want to be the person who can see crystal clear what is going on, what their options are, and who doesn’t hesitate to take them when appropriate. Anger, like flinching in general, is best as a transient event when we get surprised, and until we can reorient.
Heh. This is where we’re going to differ big time. There’s a gigantic inferential chasm here so none of this will ring true, but nevertheless here is my stance:
It is our outer wrappings that are very stupid most of the time. Even when your inner part is stupid too, that’s generally still the fault of the outer wrapping getting in the way and not doing its job. Stupid outer layer notwithstanding, that inner layer can quite easily be questioned directly. And updated directly.
The way I came to this view is by learning hypnosis to better fix these “irrational inner parts”, peeling the onion a layer at a time to come up with methods that work in increasingly diverse circumstances, and eventually recognizing that the things that work most generally are actually what we do by default—until we freak out and make up stories about how we “can’t” and “need hypnosis to fix these non-directly accessible very stupid inner parts”. Turns out those stories aren’t true, and “hypnosis” is an engineered way of fighting delusion with delusion. The stories feel true due to confirmation bias, until you tug at the seams.
It seems to me that you should change your behavior as circumstances change, even if the changes are completely expected. When you step into deep water, you should start swimming; when you step out of the water, you should stop trying to swim and start walking again. This remains true even if the changes are 100% expected.
Do you mean to say that you have some empirical way of measuring these “prediction errors” that you’re referring to, separately from the emotions you claim they explain?
Got any data you can share?
If you use your technique on an 8-year-old who is scared of the dark at night, do you actually predict your technique would reveal that they have a prediction that it won’t get dark at night? Would your technique allow you to “directly update” the 8yo so that they stop being scared of the dark?
Yes, your behavior at time t = 0 and time t = 1 ought to be different even if the changes between these times are entirely predicted. But at t = 0, your planned behavior for t = 1 will be swimming if you foresee the drop off. If you don’t see the drop off, you get that “Woah!” that tells you that you need to change your idea of what behavior is appropriate for t >=1.
I guess I should have said “Notice how your planned behavior has to change”.
Well, if you were to walk outside and get rained on, would you experience surprise? If you walked outside and didn’t get rained on, would you feel surprised? The answers here tells you what you’re predicting.
No, I wouldn’t expect the 8-year-old to be doing “I expect it to not get dark”, but rather something more like “I expect to be able to see a lack of monsters at all times”—which obviously conflicts with the reality that they cannot when the lights are out.
The way I’d approach this depends on the specific context, but I generally would not want to directly update the kids beliefs in any simple sort of way. I take issue with the assumption that fear is a problem in the first place, and generally find that in any case remotely like this, direct overwriting of beliefs is a bad thing.
I’m 13 posts into a big sequence laying out my thoughts on this, and it’s full of examples where I’ve achieved what might seem like unusual results from a “this stuff is unconscious and hard” perspective, but which aren’t nearly so impressive once you see behind the curtain.
The one I posted today, for example, shows how I was able to get both of my daughters to be unafraid of getting their shots when they were two years old (separate instances, not twins), and how the active ingredient was “not giving a shit if they’re afraid of their shots”.
If you want more direct proof that I’m talking about real things, the best example would be the transcript where I helped someone greatly reduce his suffering from chronic pain through forum PMs, following the basic idea of “Obviously pain isn’t a problem, but this guy sure seems to think it is, so how is he going wrong exactly?”. That one did eventually overwrite his felt experience of the pain being a bad thing (for the most part), but it wasn’t so quick and direct because like with the hypothetical scared 8 year old, a direct overwrite would have been bad.
If you want to learn more about “direct overwriting”, then that’s the section on attention, and I explain how I was able to tell my wife to constrict her blood vessels to stop bleeding in about thirty seconds, and why that isn’t nearly as an extraordinary claim as it might seem like it should be.
I should probably throw together a sequence page, but for now they’re all on my user profile.
I feel like I have experienced a lot of negative emotions in my life that were not particularly correlated with a feeling of surprise. In fact, I can recall feeling anger about things where I literally wrote down a prediction that the thing would happen, before it happened.
Conversely, I can recall many pleasant surprises, which involved a lot of prediction error but no negative emotions.
So if this is what you are relying on to confirm your theory, it seems pretty disconfirmed by my life experience. And I’m reasonably certain that approximately everyone has similar observations from their own lives.
I thought this was understood, and the only way I was taking your theory even mildly seriously was on the assumption that you meant something different from ordinary surprise.
I find it quite plausible they would have a preference for seeing a lack of monsters. I do not find it remotely plausible that they would have a prediction of continuously being able to see a lack of monsters. That is substantially more stupid than the already-very-stupid example of not expecting it to get dark.
Are you maybe trying to refer to our models of how the world “should” work, rather than our models of how it does work? I’m not sure exactly what I think “should” is, but I definitely don’t think it’s the same as a prediction about what actually will happen. But I could maybe believe that disagreements between “should” and “is” models play a role in explaining (some) negative emotions.
I am not searching through everything you’ve ever written to try to find something that matches a vague description.
I feel like we’ve been talking for quite a while, and you are making extraordinary claims, and you have not presented ANY noteworthy evidence favoring your model over my current one, and I am going to write you off very soon if I don’t see something persuasive. Please write or directly link some strong evidence.
Ah, that’s what you’re getting at.
Okay, so for example, say you angrily tell your employee “I expect you to show up on time!”. Then, he doesn’t, and you’re not surprised. This shows that you (meta) expected your (object level) expectation of “You will show up on time!” to be false. You’re not surprised because you’re not learning anything, because you’ve chosen not to. Notice the hesitance to sigh and say “Well, I guess he is not going to show up on time”?
This stickiness comes from the desire to control things combined with a lack of sophisticated methods of control. When you accept “He is not going to show up on time”, you lose your ability to tell him “I expect you to show up on time!” and with it your ability too put pressure on him to be punctual. Your setpoint that you control to is your expectation, so if you update your expectation then you lose your ability to (crudely) attempt to control the person’s behavior. Once you learn more sophisticated methods of control, the anger no longer serves a purpose so you’re free to update your expectations to match reality. E.g. “I don’t know if you’re going to show up on time, but I do know that if you don’t, you will be fired! No hard feelings either way, have a nice day :)”
This is a really tricky equivalence to wrap ones mind around, and it took me years to really understand even after I could see that there was something there. I explain this more in my post expectations=intentions=setpoint, and give examples of how more sophisticated attempts to control cede immediate reality and attempt to control towards trajectories instead—to concretely better results.
Yeah, I think positive emotions generally require prediction errors too, though I’m less solid on this one. People are generally more willing to update on pleasant surprises so that prediction error discomfort is less likely to persist enough to be notable, though it’s worth noting that this isn’t always the case. Imposter syndrome is an example where people get quite uncomfortable because of this refusal.
The prediction error is not the same as negative emotions. Prediction error is the suffering that happens while you refuse to update, while negative emotions like sadness come while you update. You still have to have erred in order to have something sad to learn, but it’s not the same thing.
Now that I say it, I realize I had the opportunity to clarify earlier, because I did notice that this was a point of confusion, and I chose not to take the opportunity to. I think I see why I did, but I shouldn’t have, and I apologize.
Again, giant chasm of inferential distance. You’re not passing the ITT yet, and until you do it’s going to be real tough for you to “test” anything I’m saying, because you will always be testing a misinterpretation. It’s obviously reasonable to be suspicious of such claims as attempts to hide from falsifiability, but at the same time sometimes that’s just how things are—and assuming either way is a poor way of finding truth.
To distinguish, you want to look for signs of cognitive dissonance, not merely things that you disagree with. Because if you conclude that you’re right on the surface level because the other person gets something wrong two levels deep… and your judgment of whether they’re wrong is your perspective, which they disagree with… then you’ve just given up on ever learning anything when there’s a disagreement three or more levels deep. If you wait to see signs that the person is being forced to choose between changing their own mind or ignoring data, then you have a much more solid base.
That, and look for concrete predictions that both sides can agree on. For example, you took the stance that anger was appropriate because without it you become susceptible to murderers and don’t retaliate—but once I pointed out the alternative, I don’t think you doubt that I was actually able to fight back without anger? Or is that genuinely hard to believe for you?
Hey, kids are stupid. Adults too. Sometimes people even keep expecting people to not piss them off, even when they know that the person will piss them off :p
Jokes aside, this is still the “expectations=intentions” thing. We try to not see our expectations as predictions when we’re using them as intents, but they function as predictions nonetheless.
“Should” is used as an attempt to disconnect ourselves from updating on what will happen in order to try to make something happen—because we recognize that it will likely fail and want to try anyway. If I say “You will show up on time” as if it’s a matter of fact, that’s either powerful… or laughable. And if I sense that I don’t have the authority to pull that off, I’m incentivized to back off to “You SHOULD show up on time” so that I don’t have to accept “I guess you won’t show up on time, huh?” when you don’t. I can always say “Okay maybe he won’t BUT HE SHOULD” and immediately negate the uncomfortable reality.
So “Yes, I’m talking about our models of how the world should work”, and also that is necessarily the same as our models of how the world does work—even if we also have meta models which identify the predictable errors in our object level models and try to contain them.
Maybe that part could use more emphasis. Of course we have meta models that contradict these obviously wrong object level models. We know that we’re probably wrong, but on the object level that doesn’t make us any less wrong until we actually do the update.
That’s fine, no pressure to do anything of course. For what it’s worth though, it’s very clearly labeled. There’s no way you wouldn’t recognize at a glance.
I don’t think that’s fair. For one, your model said you need anger in order to retaliate, and I gave an example of how I didn’t need anger in order to retaliate. I think the fact that I don’t always struggle with predictable anger while simultaneously not experiencing your predicted downsides is clear evidence, do you not?
Of course, this isn’t strong evidence that I’m right about anything else, but it’s absolutely fatal to the idea that your model accurately depicts the realm of possibility. If your model gets one thing this wrong, this unexpectedly, how can you trust it to tell you what else to view as “extraordinary”?
You’re welcome to read my posts, or not. They’re quite long and I don’t expect you to read them, but they’re there if you want a better understanding of what I’m talking about.
Either way, I’m happy to continue because I can see that you’re engaging in good faith even though you’re skeptical (and maybe a bit frustrated?), and I appreciate the push back. At the end of the day, neither my ability to retaliate without anger, nor my ability to help kids overcome fear by understanding their predictions, hinge on you believing in it.
At the same time, I’m curious if you’ve thought about how it looks from my perspective. You’ve written intelligent and thoughtful responses which I appreciate, but are you under the impression that anything you’ve written provides counter-evidence? Do you picture me thinking “Yes, that’s what I’m saying” before you argue against what you think I’m saying?
I didn’t respond to this because I didn’t see it as posing any difficulty for my model, and didn’t realize that you did.
I don’t think you need anger in order to retaliate. I think anger means that the part of you that generates emotions (roughly, Kahneman’s system 1) wants to retaliate. Your system 2 can disagree with your system 1 and retaliate when you’re not angry.
Also, your story didn’t sound to me like you were actually retaliating. It sounded to me like you were defending yourself, i.e. taking actions that reduced the other guy’s capability of harming you. Retaliation (on my model) is when you harm someone else in an effort to change their decisions (not their capabilities), or the decisions of observers.
So I’m quite willing to believe the story happened as you described it, but this was 2 steps removed from posing any problem to my model, and you didn’t previously explain how you believed it posed a problem.
I also note that you said “for one” (in the quote above) but then there was no number two in your list.
I do see a bunch of signs of that, actually:
I claimed that your example of your friend being afraid until their harness broke seems to be better explained by my model than yours, because that would be an obvious time for the recommended action to change but a really weird time for his prediction error to disappear. You did not respond to this point.
I claimed that my model has an explanation for how different negative emotions are different and why you experience different ones in different situations, and your model seemingly does not, and this makes my model better. You did not respond to this point.
I asked you if you had a way of measuring whatever you mean by “prediction error”, so that we could check how well the measurements fit your model. You told me to use my own feelings of surprise. When I pointed out that doesn’t mach your model, you said that you meant something different, but didn’t clarify what you meant, and did not provide a new answer to the earlier question about how you measure “prediction error”. This looks like you saying whatever deflects the current point without keeping track of how the current point is related to previous points.
Note that I don’t actually need to understand what you mean in order for the measurement to be interesting. You could hand me a black box and say “this measures the thing I’m talking about” and if the black box produces measurements that correlate with your predictions that would be interesting even if I have no clue how the black box works (as long as I don’t see an uninteresting way of deriving your predictions from its inputs). But you haven’t done this, either.
I gave an example where I made an explicit prediction, and then was angry when it came true. You responded by ignoring my example and substituting your own hypothetical example where I made an explicit prediction and then was angry when it was falsified. This looks like you shying away from examples that are hard for your theory to explain and instead rehearsing examples that are easier.
You have claimed that there’s evidence in your other writing, but have refused to prioritize it so that I can find your best evidence as quickly as possible. This looks like an attempt to dissuade me from checking your claims by maximizing the burden of effort placed on me. In a cooperative effort of truth-seeking, you ought to be the one performing the prioritization of your writing because you have a massive advantage in doing so.
Many of your responses seem like you are using my points to launch off on a tangent, rather than addressing my point head-on.
This seems like it’s just a simple direct contradiction. You’re saying that model X and model Y are literally the same thing, but also that we keep track of the differences between them. There couldn’t be any differences to track if they were actually the same thing.
I also note that you claimed these are “necessarily” the same, but provided no reasoning or evidence to back that up; it’s just a flat assertion.
There are some parts of your model that I think I probably roughly understand, such as the fact that you think there’s some model inside a person making predictions (but it’s not the same as the predictions they profess in conversation) and that errors in these predictions are a necessary precondition to feeling negative emotions. I think I can describe these parts in a way you would endorse.
There are some parts of your model that I think I probably don’t understand, like where is that model actually located and how does it work.
There are some parts of your model that I think are incoherent bullshit, like where you think “should” and “is” models are the same thing but also we have a meta-model that tracks the differences between them, or where you think telling me to pay attention to my own feelings of surprise makes any sense as a response to my request for measurements.
I don’t think I’ve written anything that directly falsifies your model as a whole—which I think is mostly because you haven’t made it legible enough.
But I do think I’ve pointed out:
several ways in which my model wins Bayes points against yours
several ways that your model creates more friction than mine with common-sensical beliefs across other domains
several ways in which your own explanations of your model are contradictory or otherwise deficient
that there is an absence of support on your side of the discussion
I don’t think I require a better understanding of your model than I currently have in order for these points to be justified.
You’re extending yourself an awful lot of charity here.
For example, you accuse me of failing to respond to some of your points, and claim that this is evidence of cognitive dissonance, yet you begin this comment with:
Are you really unable to anticipate that this is very close to what I would have said, if you had asked me why I didn’t respond to those things? The only reason that wouldn’t be my exact answer is that I’d first point out that I did respond to those things, by pointing out that your arguments were based on a misunderstanding of my model! This doesn’t seem like a hard one to get right, if you were extending half the charity to me that you extend yourself, you know? (should I be angry with you for this, by the way?)
As to your claim that it doesn’t pose difficulty to your model, and attempts to relocate goal posts, here are your exact words:
This is wrong. It is completely normal to not feel anger, and retaliate, when you have accurate models instead of clinging to inaccurate models, and I gave an example of this. Your attempt to pick the nit between “incapacitation” vs “dissuasion” is very suspect as well, but also irrelevant because dissuasion was also a goal (and effect) of my retaliation that night. I could give other examples too, which are even more clearly dissuasion not incapacitation, but I think the point is pretty clear.
And no, even with the relocated goalposts your explanation fails. That was a system 1 decision, and there’s no time for thinking slow when you’re in the midst of something like that.
No, I made it very clear. If you have a fraction of the interest it would take to read the post and digest the contents, you would spend the ten seconds needed to pull up the post. This is not a serious objection.
Again, it’s totally understandable if you don’t want to take the time to read it. It’s a serious time and effort investment to sit down and not only read but make sense of the contents, so if your response were to be “Hey man, I got a job and a life, and I can’t afford to spend the time especially given that I can’t trust it’ll change my mind”, that would be completely reasonable.
But to act like “Nope, it doesn’t count because you can’t expect me to take 10 seconds to find it, and therefore must be trying to hide it” is.… well, can you see how that might come across?
So if I tell you that the bottle of distilled water with “Drinking water” scribbled over the label contains the same thing as the bottle of distilled water that has “coolant” scribbled on it… and that the difference is only in the label… would you understand that? Would that register to you as a coherent possibility?
I’m sorry, but I’m having a hard time understanding which part of this is weird to you. Are you really claiming that you can’t see how to make sense of this?
You’re missing the point of my question. Of course you think you’ve pointed that stuff out. I’m not asking if you believe you’re justified in your own beliefs.
There are a lot of symmetries here. You said some thing’s that [you claim] I didn’t respond to. I said some things which [I claim] you didn’t respond to. Some of the things I say strike you as either missing the point or not directly responding to what you say. A lot of the things that you’ve said strike me in the same way. Some of the my responses [you claim] look like cognitive dissonance to you. Some of your responses [I claim] look that way to me. I’m sure you think it’s different because your side really is right, and my side really is wrong. And of course, I feel the same way. This is all completely normal for disagreements that run more than a step or two deep.
But then you go on to act like you don’t notice the symmetry, as if your own perspective objectively validates your own side. You start to posture stuff like “You haven’t posted any evidence [that I recognize]” and “I’m gonna write you off, if you don’t persuade me”, with no hint to the possibility that there’s another side to this coin.
The question is, do you see how silly this looks, from my perspective? Do you see how much this looks like you’re missing the self awareness that is necessary in order to have a hope of noticing when you’re inhabiting a mistaken worldview, which pats itself on the back prematurely?
Because if you do, then perhaps we can laugh about our situation together, and go about figuring out how to break this asymmetry. But if you don’t, or if you try to insist “No, but my perspective really is better supported [according to me]”, the symmetry is already broken.
You complain that I failed to anticipate that you would give the same response as me, but then immediately give a diametrically opposed response! I agreed that I didn’t respond to the example you highlighted, and said this was because I didn’t pick up on your implied argument. You claim that you did respond to the examples I highlighted. The accusations are symmetrical, but the defenses are very much not.
I did notice that the accusations were symmetrical, and because of that I very carefully checked (before posting) whether the excuse I was giving myself could also be extended to you, and I concluded definitively that it couldn’t. My examples made direct explicit comparisons between my model and (my model of) your model, and pointed out concrete ways that the output of my model was better; it seems hugely implausible you failed to understand that I was claiming to score Bayes points against your model. Your example did not mention my model at all! (It contrasts two background assumptions, where humans are either always nice or not, and examines how your model, and only your model, interacts with each of those assumptions. I note that “humans are always nice” is not a position that anyone in this thread has ever defended, to my knowledge.)
And yes, I did also consider the meta-level possibility that my attempt to distinguish between what was said explicitly and what wasn’t is so biased as to make its results useless. I have a small but non-zero probability for that. But even if that’s true, that doesn’t seem like a reason to continue the argument; it seems like proof that I’m so hopeless that I should just cut my losses.
I considered including a note in my previous reply explaining that I’d checked if you could use my excuse and found you couldn’t, but I was concerned that would feel like rubbing it in, and the fact that you can’t use my excuse isn’t actually important unless you try to use it, and I guessed that you wouldn’t try. (Whether that guess was correct is still a bit unclear to me—you offer an explanation that seems directly contradictory to my excuse, but you also assert that you’re saying the same thing as me.)
If you are saying that I should have guessed the exact defense you would give, even if it was different from mine, then I don’t see how I was supposed to guess that.
If you are saying that I should have guessed you would offer some defense, even if I didn’t know the details, then I considered that moderately likely but I don’t know what you think I should have done about it.
If I had guessed that you would offer some defense that I would accept then I could have updated to the position I expected to hold in the future, but I did not guess that you’d have a defense I would accept; and, in fact, you don’t have one. Which brings us to...
(re-quoted for ease of reference)
I have carefully re-read the entire reply that you made after the comment containing the two examples I accused you of failing to respond to.
Those two examples are not mentioned anywhere in it. Nor is there a general statement about “my examples” as a group. It has 3 distinct passages, each of which seems to be a narrow reply to a specific thing that I said, and none of which involve these 2 examples.
Nor does it include a claim that I’ve misapplied your model, either generally or related to those particular examples. It does include a claim that I’ve misunderstood one specific part of your model that was completely irrelevant to those two examples (you deny my claim that the relevant predictions are coming from a part of the person that can’t be interrogated, after flagging that you don’t expect me to follow that passage due to inferential distance).
Your later replies did make general claims about me not understanding your model several times. I could make up a story where you ignored these two examples temporarily and then later tried to address them (without referencing them or saying that that was what you were doing), but that story seems neither reasonable nor likely.
Possibly you meant to write something about them, but it got lost in an editing pass?
Or (more worryingly) perhaps you responded to my claim that you had ignored them not by trying to find actions you took specifically in response to those examples, but instead by searching your memory of everything you’ve said for things that could be interpreted as a reply, and then reported what you found without checking it?
In any case: You did not make the response you claimed that you made, in any way that I can detect.
Communication is tricky!
Sometimes both parties do something that could have worked, if the other party had done something different, but they didn’t work together, and so the problem can potentially be addressed by either party. Other times, there’s one side that could do something to prevent the problem, but the other side basically can’t do anything on their own. Sometimes fixing the issue requires a coordinated solution with actions from both parties. And in some sad situations, it’s not clear the issue can be fixed at all.
It seems to me that these two incidents both fall clearly into the category of “fixable from your side only”. Let’s recap:
(1) When you talked about your no-anger fight, you had an argument against my model, but you didn’t state it explicitly; you relied on me to infer it. That inference turned out to be intractable, because you had a misunderstanding about my position that I was unaware of. (You hadn’t mentioned it, I had no model that had flagged that specific misunderstanding as being especially likely, and searching over all possible misunderstandings is infeasible.)
There’s an obvious, simple, easy, direct fix from your side: State your arguments explicitly. Or at least be explicit that you’re making an argument, and you expect credit. (I mistook this passage as descriptive, not persuasive.)
I see no good options from my side. I couldn’t address it directly because I didn’t know what you’d tried to do. Maybe I could have originally explained my position in a way that avoided your misunderstanding, but it’s not obvious what strategy would have accomplished that. I could have challenged your general absence of evidence sooner—I was thinking it earlier, but I deferred that option because it risked degrading the conversation, and it’s not clear to me that was a bad call. (Even if I had said it immediately, that would presumably just accelerate what actually happened.)
If you have an actionable suggestion for how I could have unilaterally prevented this problem, please share.
(2) In the two examples I complained you didn’t respond to, you allege that you did respond, but I didn’t notice and still can’t find any such response.
My best guess at the solution here is “you need to actually write it, instead of just imagining that you wrote it.” The difficulty of implementing that could range from easy to very hard, depending on the actual sequence of events that lead to this outcome. But whatever the difficulty, it’s hard to imagine it could be easier to implement from my side than yours—you have a whole lot of relevant access to your writing process that I lack.
Even assuming this is a problem with me not recognizing it rather than it not existing, there are still obvious things you could do on your end to improve the odds (signposting, organization, being more explicit, quoting/linking the response when later discussing it). Conversely, I don’t see what strategy I could have used other than “read more carefully,” but I already carefully re-read the entire reply specifically looking for it, and still can’t find it.
I understand it’s possible to be in a situation where both sides have equal quality but both perceive themselves as better. But it’s also possible to be in a situation where one side is actually better and the other side falsely claims it’s symmetrical. If I allowed a mere assertion of symmetry from the other guy to stop me from ever believing the second option, I’d get severely exploited. The only way I have a chance at avoiding both errors is by carefully examining the actual circumstances and weighing the evidence case-by-case.
My best judgment here is that the evidence weighs pretty heavily towards the problems being fixable from your side and not fixable from my side. This seems very asymmetrical to me. I think I’ve been as careful as I reasonably could have been, and have invested a frankly unreasonable amount of time into triple-checking this.
Before I respond to your other points, let me pause and ask if I have convinced you that our situation is actually pretty asymmetrical, at least in regards to these examples? If not, I’m disinclined to invest more time.
Oh, the situation is definitely asymmetrical. In more ways than you realize.
However, the important part of my comment was this:
If you can’t say “Shoot, I didn’t realize that”, or “Heh, yeah I see how it definitely looks more symmetrical than I was giving credit for (even though we both know there are important dissymmetries, and disagree on what they are)”, and instead are going to spend a lot of words insisting “No, but my perspective really is better supported [according to me]”… after I just did you the favor of highlighting how revealing that would be… then again, the symmetry is already broken in the way that shows which one of us is blind to our limitations.
There’s another asymmetry though, which has eluded you:
Despite threatening to write me off, you still take me seriously enough to write a long comment trying to convince me that you’re right, and expect me to engage with it. Since you failed to answer the part that matters, I can’t even take you seriously enough to read it. Ironically, this would have been predictable to you if not for your stance on prediction errors, Lol.
Also, with a prediction error like that, you’re probably not having as much fun as I am, which is a shame. I’m genuinely sorry it turned out the way it did, as I was hoping we’d get somewhere interesting with this. I hope you can resolve your error before it eats at you too much, and that you can keep a sense of humor about things :)
Guess we’re done, then.
We can be, if you want. And I certainly wouldn’t blame you for wanting to bail after the way I teased you in the last comment.
I do want to emphasize that I am sincere in telling you that I hope it doesn’t eat at you too much, and that I hoped for the conversation to get somewhere interesting.
If you turn out to be a remarkably good sport about the teasing, and want to show me that you can represent how you were coming off to me, I’m still open to that conversation. And it would be a lot more respectful, because it would mean addressing the reason I couldn’t take your previous comment seriously.
No expectations, of course. Sincere best wishes either way, and I hope you forgive me for the tease.