Deconfusing Regret
There are lots of different things, all of which are called regret. I treat them differently, so I need different names for the different classes of regret.
Agentic regret. The sort of regret that you feel when you made a choice that was wrong and you knew it was the wrong choice and you failed as an agent. You knew you should have taken out the trash bins, but you didn’t, and now it’s the next day and you have full bins and no way to deal with them until next week. Hope you don’t need to do anything that produces trash.
Bad-randomness regret. You made a choice, and you knew it was a gamble, and you lost, even though the gamble had positive expected value. Even if you know your action had positive value, if you had the information while you made the decision that you had now, you could have made a better decision. If you could have gained the information and you knew that ahead of time, this is agentic regret—failing to gather the information required to make a good decision is a bad decision.
Self-knowledge regret. You made a choice, and got what you thought you wanted, but it turns out you didn’t want that thing. This may be revelation regret if you knew that you didn’t know what you wanted. But sometimes this comes as a surprise—you get to the top of that mountain and think “whatever, this sucks, I want to go home”, or you finish that book you’ve been forcing yourself to read and think “whatever, this sucks, I don’t want to be at home”.
Temporal-conflict regret. The version of you in the past that made a decision (warning: real philosophy happening, “what is the self?”, panic and flee). That decision was good for them but is bad for you, the version of you that exists now. In other words, some asshole (your past self) did something unkind to you (made a decision that benefitted them at the cost of you). Procrastination is a good example of this. If you spend a day on disposable pleasures at the cost of avoiding your meaningful pursuits, the you who has to clean up … your …. mess will be upset with … you … or whoever. Is this agentic regret? Not always. If you plan to spend the next two days doing one day of work and one day of leisure, it is selfishness, not failure, that causes the you of now to saddle the you of the future with the day of leisure. (Or is it selfishness for the you of the future to demand the day of leisure over the day of work?) Regretting a relationship after a breakup is often this: the you of today has to deal with the fallout of the breakup, but the you of the past got to enjoy the relationship while it existed, the selfish bastard.
Regret is a form of suffering, it is a negative-valence emotion, and negative-valence emotions are only worth feeling if they motivate choices that you endorse.
Agentic regret is worth feeling, because knowing that you will suffer if you choose poorly motivates you to choose wisely.
Bad-randomness regret is worth destroying, because knowing that you will taking positive expected value gambles motivates you to miss good opportunities.
Self-knowledge regret is worth pondering, because the value of learning more about your utility function is often worth much more than the bad decision, but the cost of working towards the wrong goal is also often very high.
Temporal-conflict regret reflects a lack of unity between your selves. It is worth getting to a point where you are unified in your desires and you understand, on a level where it has become an inseparable part of you, that every action you take represents a policy of always taking that action in that situation. Then make policies that minimize inter-self competition.
Feel the regrets that it helps you to feel, and have equanimity about the situations that would otherwise cause regret that is unhelpful.
I have no regrets after reading your post. Thank you namebro
I don’t feel regret. I used to, but with a lot of work I came to trust that I’m always doing my best, so I can never have regret because I couldn’t have done anything else. I can’t even regret that my best was not enough because if I could have made a better choice when I had to make the choice I would have. No point getting worked up over things beyond my control. Just learn from it and do better next time.
That said, you can’t just force your way to this position, and doing so would probably be bad. Regret, as you note, is useful for a lot of people. It’s load bearing for them, part of the complex tangle of confusion and disfunction that’s been carefully balanced to keep their lives going. It’s not a great state to be in, but it works, and it’s scary to imagine pulling out regret because doing so without fixing a lot of other things first would make the whole system collapse by removing a key source of motivation.
To clarify, is this
Something you claim about yourself specifically, presumably unusually among humans? (I certainly don’t think that I’m always doing my best.)
If so… it’s admittedly rude to question someone’s self-reports; nevertheless, I confess skepticism.
Something you claim about the concept of “doing one’s best”, such that humans in general are basically always doing their best?
If so, this strikes me as an unhelpful use of the term.
Something else?
The second. In fact, it’s bigger than that. Everything that optimizes is always doing its best, and also its worst, and its median, etc.
This phrase is meant to point out the contradictory nature of the idea of “best” that we strive for. Subjectively, we reason about causality and see counterfactuals that were worse and so judge what’s best against those counterfactuals. But there’s also an absolute sense of “best” that is meaningless because we’re not actually in control but rather observers within the universe seeing how things play out and identifying with some chunk of the universe that’s heavily intertwined with the process that produces our observations (even this is a bit wrong because it’s making some implicit claims about what this observer is that I wouldn’t endorse, but I think this is a useful enough shorthand for this explanation).
Thinking about the subjective, counterfactual best is sometimes useful as a side effect of how we figure things, but in the context of speaking about how the feeling of regret is optional it makes sense to emphasize the latter sense in which best/worst/etc. has no real meaning.
Yeah, my reaction is still that this is just a completely unhelpful way of using the term. Like sure, if someone is confused about determinism then it seems good to deconfuse them on that score. But my sense is that
Most people who say “I didn’t do my best” probably aren’t saying that because they’re confused about determinism.
Telling them “you did do your best, because there was nothing else you could have done, so you’re always doing your best” is more likely to confuse them about agency than to deconfuse them about determinism.
I find as a coach that sometimes people really need the view of determinism (often framed as radical acceptance), and they in some sense don’t alieve the thing Gordon is talking about.
At other times, they are confused about agency, (often framed as radical responsiblity) and they don’t alieve the thing you’re talking about.
In either case, I usually frame each way of seeing as a skill, and tell them we’re not trying to get rid of their current way of seeing, just add another option. And try to get at when and why each might be useful.
I’d be curious to hear examples of this. If I try to come up with them myself, they have a flavor of “lying to people to make them feel better”. (Which I’m not gonna say is categorically bad, but I feel like if you happen to agree that it has that flavor then that seems like a good thing to note on LW. But maybe you just don’t agree it has that flavor.)
Like, I’m imagining here the type of person who’s trying to save the world by working ten hour days seven days a week, and then they have a day where they just lie in bed and play video games and they’re like ”???why didn’t I do my best???”
And I can imagine replying to that with “you did do your best, your best is all you ever do”. But that feels like lying to me. A less-lying answer to me would be something like:
To me, this rhymes with the determinism answer, but seems importantly different.
Is this at all close to what you’re thinking?
No, it’s much more like “you are hurting yourself fighting with reality”. The reality is that you slept and played video games and you’re wasting mental energy fighting something that:
You can’t change anymore.
You couldnt even have changed them, because you didnt have this thought process/coaching session/emotional state, etc.
So first, let’s accept the state of things as they are, and have compassion for the version of you that didn’t have the tools or ability to make a different decision.
Having done that, let’s have compassion for present you by learning from past you, and mentally practice what you could have done in this situation (or well before this situation) to avoid burnout. This way, future you will have those tools.
Nod. But I note that “you couldn’t have changed this back then, because you didn’t at the time have the tools to do this” seems very different from what I think Gordon is saying?
Like, I see a big difference between “there was only one thing you could have done, and you did it” and “this specific thing that you might in theory have done, was practically unavailable to you”.
Like, “because you didn’t at the time have the tools to do this” admits that we can classify past actions-not-taken into “those we could have taken but didn’t” and “those we couldn’t have taken” (perhaps “those we couldn’t have taken for reason X”, “those we couldn’t have taken for reason Y”, …), whereas Gordon seems to want to classify all past actions as simply “those we could not have taken”.
To me the argument is:
You literally couldn’t have done anything different in the past.
But you can IMAGINE what you would have done differently in the past in order to affect the future.
I will often invoke the idea that you literally did the best you could at the time when walking people through this.
Um. So do you not see a big difference between
A: “there was only one thing you could have done, and you did it”
B: “this specific thing that you might in theory have done, was practically unavailable to you”
?
To me, this conversation feels like… you said you tell people B, and I said that seems very different from A to me, and now you’re saying you tell people A? And I’m not sure if you’re saying that to clarify “oh, no, I don’t tell people B, I tell them A”, or because you don’t see the distinction I’m trying to draw, or what. I’m not really sure where to go from here.
I’ll say that A is the thing that feels to me like lying to people, in that it seems true but only in an irrelevant sense. This comment might help clarify?
“You couldnt even have changed them, because you didnt have this thought process/coaching session/emotional state, etc.” is ambiguously either A or B. And I often explain it as A.
But “you didn’t have this thought process/coaching session/emotional state, etc.” isn’t a crux for A, right? I feel like if what you mean is A, then giving that reason is violating a norm of communication, and so it doesn’t particularly feel ambiguous between A and B to me.
But, okay, thanks. I think I see what you’re saying.
This is a tangent, but I’m curious if you agree with me that telling people A feels like lying to make them feel better? (No need to try and justify it or anything if so.)
No, it definitely isn’t lying.
That someone would say that they did their best is considerable evidence that they are confused about determinism because if they actually understood it they would know that they couldn’t have done anything else, and thus remarking that they didn’t do their best suggests they still believe the world could have been otherwise.
I’m not sure that they’re likely to get more confused about agency since it seems like nearly everyone is confused about just how to define agency in a way that both conforms with intuitions and doesn’t rely on things like metaphysical claims to free will.
I’m confused. Previously you said
So it sounded to me like you understand what I would say is the normal meaning of the phrase “I didn’t do my best”, you just think that in this particular case it’s helpful to use some other meaning; where I claim that that other meaning is unhelpful.
But from this reply, it sounds like… you don’t understand what I would say is the normal meaning? Or you think it’s a niche meaning, such that someone who says “I didn’t do my best” is more likely to be confused about determinism than to be using that meaning?
Both of those would be surprising to me.
(I am admittedly confused about agency, and I agree that probably most people are. But it still seems entirely possible to make someone more confused or less confused, so “most people are confused about agency, so I’m unlikely to make them more confused” doesn’t follow.)
Roughly, yes. “[N]iche” doesn’t quite make sense here. I might say it’s more like the naive meaning. If you really get that the things couldn’t have been otherwise you’d say something different from “I didn’t do my best” like “next time I want to do better” or “I have a lot to learn” that expresses roughly the same meaning as counterfactual “regret” but also making it clear that in that past instance it wasn’t possible to have done anything differently.
This just doesn’t seem true to me. Also, neither “next time I want to do better” nor “I have a lot to learn” seems like a suitable replacement, and “I didn’t do my best” doesn’t seem synonymous with “regret” although they overlap in circumstance.
My current sense is that you don’t understand what I consider the normal meaning of the phrase, but also my model of the conversation is that if I try to explain you’ll reply with what I consider to be gotchas that completely miss the point.
(And it’s not super easy to explain because yeah, it involves agency and I’m confused about agency. But I predict that it all adds up to normality, and that our disagreement here does not come from you understanding agency better than I do.)
I’m happy to drop the conversation, but if you want to continue it (it’s fine if you don’t), I’d suggest one way to continue might be for you to try to explain what I think the normal meaning is, and why I think it’s valuable to keep around rather than e.g. trying to replace it with the things you suggest.
Quick edit, as a note: at this point I’m not saying “the way you use the phrase is bad”. Like, that kind of is a thing I think, and my first two comments were indeed criticizing your use of it, but if you were like “yeah it’s kind of weird but I think it’s valuable” then I don’t think I’d have much to say to that. But the conversation shifted to you not simply defending your use, but criticizing other uses (“That someone would say that they did their best is considerable evidence that they are confused about determinism”, “If you really get that the things couldn’t have been otherwise you’d say something different”), and that seems completely wrongheaded to me.
Sure, I totally get why people typically say “I didn’t do my best”. It’s a recognition of a counterfactual world that wasn’t realized: if they had counterfactually done something else they reckon that the outcome would have been better than what they got. It also expresses a wish, something like “I wish I had done something else that had a better result”. And there’s also a bit of something that I’d consider reasonable: an expression that you didn’t do as well as you’d expect against your base rate of performance in similar situations.
I also get that saying this is psychologically necessary for many people given how they reason about counterfactuals, and that they wouldn’t actually get better at things unless they were able to think about counterfactual pasts and pretend that they could have happened.
Thus I don’t actually think we shouldn’t keep around this meaning! Many people need it in order to live their lives. My point here is if you hold to this meaning of regret then you’re confused about causality.
Yeah, this seems to me to be missing something big. Here are some things we might say I could have done last night:
Read a book
Read The Iliad in ancient Greek
Played ukulele
Played a 10⁄10 rendition of We Are the Crystal Gems on ukulele, singing along in tune
Played theremin
Gone to the pub
Gone to Milton Keynes
Gone to France
Gone to the moon
Gone to Alpha Centauri
Turned into a small cat
Doodled some math symbols on a piece of paper
Written a proof of the Riemann Hypothesis
Drunk a liter of coffee
… and fallen straight to sleep
Successfully performed open-heart surgery
Continued playing Baba Is You
Finished Baba Is You without looking up any hints or solutions
As it happens I only did one of these. So there’s a sense in which there was only one I could have done, and it’s the one I actually did.
But there’s another, very reasonable and important sense, in which some of these are things that yes, I could have done; and some are things that no, I could not have done. I can read, but I can’t read ancient Greek. I don’t own a theremin, though maybe I could have gotten one delivered if I really wanted? I’m not good at ukulele or singing. I could probably have found a train to Milton Keynes and likely even France; but I don’t think all the money in the world would have gotten me to the moon on that short notice; and Alpha Centauri is simply too far away to go in one night. It’s plausible that there’s a proof of the Riemann hypothesis short enough for me to have written it down in an evening, but it’s not plausible that I would have produced it. And so on.
I claim that when most people talk about whether they did their best, they’re thinking about other things they could have done in this very reasonable and important sense.
Of course, it’s not always obvious whether something is a thing I could have done or not, in this sense. So sometimes it might not be clear whether I did my best or not. That’s fine. It still leaves the phrase much more useful than a phrase that means “as a being of physics, I did the only thing I possibly could have done as dictated by the evolution of the quantum waveform or whatever, which is all that anyone ever does”.
This doesn’t seem like anything different than what I said. Everything you’re talking about is a counterfactual things someone could have counterfactually done but didn’t.
… you claim that I counterfactually could have gone to Alpha Centauri last night? And that there’s no meaningful sense in which this is different from how I counterfactually could have played Baba Is You last night?
Sorry but if that’s what you’re saying I can’t take it seriously, and if it’s not what you’re saying I have no idea what you’re saying instead of that.
The only difference between these two counterfactuals is how likely we should believe each one could have happened. There is a non-zero probability you could have gone to Alpha Centauri last night, just as there is that you played Baba is You. They don’t have the same probability of having happened and one is much more likely than the other, but they are still of the same kind.
(This seems obviously wrong to me but I don’t have any hope that continuing will be productive.)
It’s up to you, but to be frank I’m also frustrated with how you’ve approached this discussion. You’ve only provided a vague and extensional explanation of what you mean by regret, whereas I’ve provided more detailed models of what I’m thinking that you’ve objected to by roughly saying “idk doesn’t match my intuitions”, which is not something we can build an actual productive discussion on unless you dig into your intuitions more and provide of model for why they are useful.
Alright, to go meta:
I disagree that you’ve provided detailed models. I don’t think I have, either; I think it’s mostly been us butting heads over “these two things are different” / “these two things are the same”.
It’s probably the case that I haven’t been very helpful. If I’m honest, I think a lot of this is because I remember multiple conversations with you where: in the beginning you seemed to me to be saying something confused; and by the end neither of us seemed to have changed our minds. (At least one of these was also a case of you saying that two obviously-to-me different things are actually the same.) I remember zero conversations with you where: in the beginning you seemed to be saying something confused; and by the end either you’d convinced me or I’d convinced you. Which doesn’t mean it’s never happened, but like. The track record of our past conversations does not bode well for our future conversations.
So when you say something that seems to me confused, I’m not sure how to react. I can just downvote/disagree/react and let it go—there’s another LW user who I do that fairly often with—but I think your confusion is often subtle enough to be worth pointing out. (Recall that this thread started when I thought you might be saying one of two things. I thought one was wrong and the other was silly but mostly harmless. It took a few messages to even reach the actual thing we’ve been disagreeing about.) I can try to approach with an open mind and consider that you might have something to teach me just as I think I have something to teach you; but that’s not what I’m actually feeling.
I think my most-common approach in situations like this, and what I went for this time, was roughly “avoid being openly antagonistic, but don’t put lots of effort in; don’t particularly expect to deconfuse you; do try to make it clear to the audience what the disagreement is about, so they can make up their own minds”. (One theoretically good thing about this is that if it does turn out to be me who’s confused, then hopefully it becomes easier for an audience member to step in and explain it to me differently. But so far this hasn’t happened, to my recollection.[1] Nor do I recall someone stepping in to agree with me and try a different approach of explaining my perspective to you.)
(On “avoid being openly antagonistic”, I’m not currently sure whether “I can’t take it seriously” was toeing the line or over it.)
My current sense is that where the thread really went wrong is when I said “if you want to continue it”. I think I was a bit surprised that you did. And you put in what seemed to me to be some effort, even though history said that you were unlikely to change my mind (it said this to me, dunno if it said it to you); and I would have felt bad if I didn’t put in a bit of effort myself, even though history said I was unlikely to change your mind. My current guess is that I should have just dropped things there.
Also, you say I haven’t been clear about what I mean by regret, but I have not been talking about regret. The only time I’ve used that word so far has been to say that it’s not what I’m talking about. I dunno if this is a particularly important detail, but it sure seems like a metaphor.
Part of me anticipates you-or-someone-else thinking that my thread with Matt Goldenberg was this. But it seems to me that Matt agrees with me, on the thing that you and I have mostly been disagreeing about.
To paraphrase the hiker’s saying “regret is mandatory, suffering is optional”. What you describe as not feeling regret to me sounds like feeling regret but not suffering because of it. Knowing that you could have made a better choice is an act of feeling regret for the choice you did make. Suffering as a result of it is bad for you (it’s suffering, after all), and it sounds like you don’t suffer when you regret. This is a good place to be! It’s good to both recognize that there were better possibilities, and maybe you can aspire to pick better next time, but maybe you did ultimately do as good as you could have done in that situation, so beating yourself up wouldn’t be useful.
We may disagree on the semantics of the word regret, so imagine I’m saying regret_{alex} for my version, and regret_{gordon} for your version.
I dispute the claim that regret is mandatory in most senses of the word.
I’m specifically saying that I could not have made a better choice because I already made the best possible choice given the circumstances, so there is nothing to regret other than the sort of “regret” that I did not counterfactually maximize expected value.
Behind this claim about regret is another: the universe is subjectively deterministic (the universe looks deterministic from the view point of an observer, and any appearance otherwise is due to uncertainty rather than free will or randomness). This claim allows us to avoid making any metaphysical (and thus unprovable) claims like that the universe is really deterministic, that free will exists, or that counterfactuals are real (as opposed to constructs to support the reckoning of causation).
This… seems like it’s not engaging with what regret is for?
Like, there is definitely a sense in which everything is deterministic. But, like, the point of regret is to learn and do different things in the future.
I’m a bit surprised at your viewpoint here given other things I knew about you, though, and not sure if I’m missing something.
My motivation with these comments is to push back against claims that regret is necessary or mandatory. I agree that regret is pretty useful for most people, and I wouldn’t recommend they give it up unless it just falls away (and insert obvious caveat about psycho/sociopaths). But I also think it’s worth knowing that feeling regret is not necessary to live a fulfilling life, because not knowing that it’s not necessary creates a roadblock where people cling to their regret long past their need for it.
It might help to say a bit more about what my experience of regret is. It used to be a feeling, like a burning sensation of fear and anxiety, like the fear that I’ll be abandoned by friends and family for screwing up. Now, after doing a bunch of meditation and other stuff to come to terms with the world as it is, I see regret like a negative number on a spreadsheet: useful information to update and act. This is different enough of an experience that I think it’s better not to call it regret, since that seems likely to lead to more confusion, since I think most people have strong associations with what the feeling of regret is like rather than what the accounting to regret is like.
It might also help to know that I see regret as separate from remorse. Whereas regret, when felt, might be taboo’d as “clinging to counterfactuals” in its various forms, remorse is more like sadness that the world is as it is and that my best did not produce a better outcome. It’d take a lot of equanimity not to feel remorse, but it takes surprisingly little equanimity to not feel regret.
Heh, I think this is the opposite of our respective roles in our previous conversation about trauma. (Where you were like “I tend to think of trauma as things that happened in the past that led to stuck memories that are strongly immune to updating.” and I was like “that seems different enough from standard usage you should probably find a new word?)
So, I’m pretty open to the “regret is different enough for most people than how I’m describing it that I should have a new word.” But, I also personally have thought of regret as fairly straightforwardly matching the way I described it. I don’t feel like I did much rationality-reimagining to end up with my description above. (i.e. most people might not say ‘the point of regret is to learn things for the future’, but I do think it sort of straightforwardly describes how people are using it’)
I like this post because it is thoughtful and well expressed. Getting old leaves one very aware of one’s errors which as you say can lead towards self knowledge so that change will, of necessity, take place. Regret is therefore a learning opportunity for me but to suffer from hindsight is not useful. I think to be informed by mistakes is as good as it gets. Whilst it is not always the case that I won’t repeat a particular error, one hopes that one is learning! So, the option of regret, and how seriously to take oneself/one’s past thoughts or actions, is useful.