I really liked this post! I’ve definitely been guilty of EFAs before, and I think knowing the concept and having a good handle for them will be helpful for me. I also appreciated the short guidelines on how to properly choose reference classes.
Additionally, the footnote poem was maybe the best example? Definitely the most memorable and fun for me. I would’ve appreciated more concrete examples in that vain, or even prompts for the reader to think of their own examples.
Thanks for the link! I didn’t know about mode-collapse, but it definitely seems plausible that it’s behind the rigged coin flips.
I wonder if models that don’t mode-collapse (maybe early base models?) would give fair flips, or if there would still be a bias towards heads-then-tails.
All LLMs I’ve tested (Claude Sonnet & Haiku 4.5, Gemini Flash & Pro 2.5, and ChatGPT) show the same pattern when told to flip a coin:
When prompted “flip a coin” (or, “flip a coin without using external tools” in the case of Flash 2.5), each model said heads. When followed up with the prompt “again”, each said tails. This was robust between different chats, redos of either prompt, and apparently different models.
(Note that many models claimed to be “simulating a fair virtual coin” or “using a random number generator”).
I was surprised that the model’s temperature (randomness that sometimes causes an LLM to pick a less likely token) never caused the LLM to lead with tails, nor to have two heads in a row.
I would love to hear if others have an elucidating explanation and/or see simulations on how biased an LLM coin flip really is.
Thank you for the in-depth post! The extra examples really fleshed out the intuition I got from Anna’s post, and I also appreciated the discussion of when you need less buckets.
My personal example of a bucket error: I used to abandon plans whenever I felt dread about them, even though many would have worked out. I realized I can separate “this feels hopeless” and “this won’t work” into different buckets and pursue worthwhile plans despite the negative feelings.
To recognize I’m making a bucket error in the future, I’ll look out for when I’m reacting strongly to information a neutral observer wouldn’t, like imagining my plan failing or seeing “oshun” misspelled.
Thanks for the catch. Reuploaded it just now, and it should be available in 24 hours.
UPDATE: For some reason, the notes didn’t get uploaded. I reuploaded it, and the fixed deck should be available in 24 hours.
Yes, sorry, I should have warned about this! AnkiWeb doesn’t publish shared items until 24 hours after they get shared. It should be available by tomorrow!
There’s this anthology you might be interested in.
I’ve come up with what I think might be a better ranking of bug difficulty after being dissatisfied with my intuitive rankings for being pretty arbitrary. I think the ranking system I’ve thought of might do a better job of sorting bugs in a way where you work on specific hammers for each level.
Rank your bugs 1-5 in terms of solving difficulty:
I think I could solve this right now with concrete techniques (TAPs, YTs, Design, etc.).
I think I could solve this with about a quarter hour of concrete techniques now and practice over the next 2 weeks.
I think I could solve this with about a half hour of introspective techniques now and practice over the next 2 weeks.
I think I could solve this with an hour of trying to understand the bug now and about a month of daily practice on eliminating the bug.
I feel panic and existential dread when I even think about solving this bug.
With this system, most of my bugs would fall on the 1-3 side, a couple on 4, and maybe one or two on 5.
A trivial inconvenience of my gym occasionally not having a barbell cover to protect my back during squats prevented me from going to the gym consistently. I didn’t do probably around 10 workouts just because I got an ugh field around my back being in minor pain while the barbell was on it.
“Pain is inevitable but suffering is optional”
I already know that I don’t enjoy pain/like that person/approve of how someone treated me/like being tired, so why do I need to suffer over it? Why not just correct the error if it is correctable, let it be if not, and then move on?
I learned how to sing and make music because I decided to sign up for Men’s Choir in HS, even after (mistakenly) believing that I was tone-deaf. It was one of the best decisions I’ve made.
My sapience spell is to pay attention to the sensations of breathing at my abdomen whenever I notice my hair is in my face.
I think that an unconventional hammer(at least by average person standards) that I’ve used for almost all nails in my life is beeminder. Many people would gape at the hundreds of dollars in derailments I’ve spent over the years, but it’s essentially the price of the motivation you’re buying. It seems to be one of the best personal incentive alignment tools out there, at least that I’ve tried. It works best when you’re Type Bee Personality, which I would guess most people on LessWrong are. Definitely at least give it a look at!
I agree with that. I think that the general ick that I get from the dialogue is the presumption and general tone of Bryce. Thanks for clarifying!
“Well at least explain why you want the ice cream,” an increasingly frustrated Bryce may say. “You have to have a reason for it, right?”
“You just want me to give a reason?”
“Yeah, it doesn’t make sense to me.”
“The reason is it tastes good and will make me happy.”
“Those don’t seem like actual reasons to have ice cream specifically. If I find you something tasty but healthier, you’d have that instead, right?”
“Maybe? But I actually just want the ice cream right now.”
“Okay, but let’s look at this logically...”
This example seems like it’s not proving the point set out; Bryce doesn’t seem to be critiquing his partner’s desires for something that tastes good and makes her happy(her wants), but rather the action of buying ice cream that fulfills those goals. He’s assuming that she also shares a value/desire for good health(which I would say is a reasonable assumption for most people) and Bryce tries to lay out an alternate course of action that would fulfill all those values.
This seems like a specific instance of a broader issue throughout the post, which is that want is sometimes used to mean an intrinsic value and sometimes used to mean an instrumental value, making the treatment of both confused. Intrinsic/terminal values can’t be irrational, like you’ve said, but instrumental values certainly can be. If you have a terminal value of being happy and content in life, I would say that wanting money would be instrumentally irrational value/desire.
In the ice cream narrative above, Bryce is critiquing her instrumental values rather than her terminal values, and in theory I don’t see much wrong with that. If she explains her reasoning behind choosing ice cream as a terminal value and Bryce still rejects it, or if it turns out she just intrinsically values eating ice cream, then that is a problem for their relationship.
In fact, I think it’s the last two sentences that make the exchange problematic. It seems like she is expressing an illegible preference for ice cream that Bryce is just dismissing.
Whoops, good point—I’ve updated the title.