High-level actions don’t screen off intent
One might think “actions screen off intent”: if Alice donates $1k to bed nets, it doesn’t matter if she does it because she cares about people or because she wants to show off to her friends or whyever; the bed nets are provided either way.
I think this is in the main not true (although it can point people toward a helpful kind of “get over yourself and take an interest in the outside world,” and although it is more plausible in the case of donations-from-a-distance than in most cases).
Human actions have micro-details that we are not conscious enough to consciously notice or choose, and that are filled in by our low-level processes: if I apologize to someone because I’m sorry and hope they’re okay, vs because I’d like them to stop going on about their annoying unfair complaints, many small aspects of my wording and facial expression will be likely different, in ways that’re hard for me to track. I may think of both actions as “I apologized politely,” while my intent nevertheless causes predictable differences in impact.
Even in the donations-from-a-distance case, there is some of this: the organization Alice donates to may try to discern Alice’s motives, and may tailor its future actions to try to appeal to Alice and others like her, in ways that have predictably different effects depending on eg whether Alice mostly wants to know/care/help or mostly wants to reinforce her current beliefs.
(This is a simple point, but I often wish to reference it, so I’m writing it up.)
More generally, it’s not just the effect that matters (globally/holistically/long-range-ish-ly speaking), but also whatever is ~causally upstream from the effect, because whatever is ~causally upstream from the effect determines what’s gonna happen when the context changes the things upstream from the effect.
Conversely, contexts where high-level actions do screen off intent are hard to construct and fragile, and sometimes valuable, therefore precious. For example, you can have a tournament arbiter who enforces the rules completely and accurately. Ze might show disdain or admiration or whatnot towards some players, but if the rules are well-constructed and ze enforces them, in principle the screening-off could be near complete. (Or not, e.g. if the non-verbals nudge players to not bring up rules objections.) On the other hand, if there’s even a small hole in the rules or probability of discretion in enforcement (as there usually is), the screening off is destroyed.
If you can make “golems” that transparently, mechanically enforce the rules, you get even more screening off. E.g. a computer-programmed game.
Mindreading is ubiquitous
Guessing the intention of others can also help predict future actions. “Alice who was court ordered to donate bed nets and doesn’t want to go to prison” vs. “Alice the EA who has read MacAskill cover to cover” have very different projected patterns of future behaviour, especially if the context changes (e.g. the court order is completed). Notably, that doesn’t stop them from both clicking the same buy button on Amazon today.
Related: Newcomb-like problems are the norm.
I think of this as coarse-grained influence vs. fine-grained influence, which basically comes down to how many bits are needed to specify the nature of the influence.
e.g., Betty could cause one more girl to have a mentor either by volunteering as a Big Sister or by donating money to the Big Sisters program.
In the case where she volunteers and mentors the girl directly, it takes lots of bits to describe her influence on the girl being mentored. If you try to stick to the actions->consequences framework for understanding her influence, then Betty (like a gamer) is engaging in hundreds of actions per minute in her interactions with the girl—body language, word choice, tone of voice, timing, etc. What the girl gets out of the mentoring may not depend on every single one of these actions but it probably does depend on patterns in these micro-actions. So it seems more natural to think about Betty’s fine-grained influence on the girl she’s mentoring in terms of Betty’s personality, motivations, etc., and how well she and the girl she’s mentoring click, rather than exclusively trying to track how that’s mediated by specific actions. If you wanted to know how the mentoring will go for the girl, you’d probably have questions about those sorts of things—“What is Betty like?”, “How is she with kids?”, etc.
In the case where Betty donates the money, the girl being mentored will still experience the mentoring in full detail, but most of those details won’t be coming directly from Betty so Betty’s main role is describable with just a few bits (gave $X which allowed them to recruit & support one more Big Sister). e.g., For the specific girl who got a mentor thanks to Betty’s donation, it probably doesn’t make any difference what facial expression Betty was making as she clicked the “donate” button, or whether she’s kind or bitter at the world. Though there are still some indirect paths to Betty influencing fine-grained details for girls who receive Big Sisters mentoring, as the post notes, since the organization could change its operations to try to appeal to potential donors like Betty.
Subliminal learning in LLMs might also be an example of this (surely a sequence of random numbers can’t convey any information about an intent to love owls, right?).
All screens are leaky.
Hm. My instinct here is that the degree to which intent matters depends a lot on the situation. Often times it matters, often times it doesn’t, often times it’s somewhere on the spectrum.
I’m having trouble figuring out how that instinct meshes with the claim that this post makes. The post says that “in the main” intent matters, but yeah, I’m not really sure what that means.
I found the apology example helpful though. It made me realize that intent in those situations matters more than I previously thought, and that being honest/genuine is an even better heuristic than I previously thought. Maybe the idea behind this post is that people tend to undervalue how much intent matters.
, consequences do.
Only if your ethics are purely utilitarian.
The more interesting question is why would anyone ever assume that to be the case, in the first place?
Unless they’ve literally never encountered a deceptive person in their life, it just seems implausible to not notice this.