I’m a 3rd year PhD student at Columbia. My academic interests lie in mechanism design and algorithms related to the acquisition of knowledge. I write a blog on stuff I’m interested in (such as math, philosophy, puzzles, statistics, and elections): https://ericneyman.wordpress.com/

# Eric Neyman

# Solving for the optimal work-life balance with geometric rationality

Note that this is just the arithmetic mean of the probability distributions. Which is indeed what you want if you believe that P is right with probability 50% and Q is right with probability 50%, and I agree that this is what Scott does.

At the same time, I wonder—is there some sort of frame on the problem that makes logarithmic pooling sensible? Perhaps (inspired by the earlier post on Nash bargaining) something like a “bargain” between the two hypotheses, where a hypothesis’ “utility” for an outcome is the probability that the hypothesis assigns to it.

The aggregation method you suggest is called logarithmic pooling. Another way to phrase it is: take the geometric mean of the odds given by the probability distribution (or the arithmetic mean of the log-odds). There’s a natural way to associate every proper scoring rule (for eliciting probability distributions) with an aggregation method, and logarithmic pooling is the aggregation method that gets associated with the log scoring rule (which Scott wrote about in an earlier post). (Here’s a paper I wrote about this connection: https://arxiv.org/pdf/2102.07081.pdf)

I’m also exited to see where this sequence goes!

Thanks for the post! Quick question about your last equation: if each h is a distribution over a coarser partition of W (rather than W), then how are we drawing w from h for the inner geometric expectation?

How much should you shift things by? The geometric argmax will depend on the additive constant.

Thanks for the post—I’ve been having thoughts in this general direction and found this post helpful. I’m somewhat drawn to geometric rationality because it gives more intuitive answers in thoughts experiments involving low probabilities of extreme outcomes, such as Pascal’s mugging. I also agree with your claim that “humans are evolved to be naturally inclined towards geometric rationality over arithmetic rationality.”

On the other hand, it seems like geometric rationality only makes sense in the context of natural features that cannot take on negative values. Most of the things I might want to maximize (e.g. utility) can be negative. Do you have thoughts on the extent to which we can salvage geometric rationality from this problem?

I wonder if the effect is stronger for people who don’t have younger siblings. Maybe for people with younder siblings, part of the effect kicks in when they have a younger sibling (but they’re generally too young to notice this), so the effect of becoming a parent is smaller.

“My probability is 30%, and I’m 50% sure that the butterfly probability is between 20% and 40%” carries useful information, for example. It tells people how confident I am in my probability.

I often talk about the “true probability” of something (e.g. AGI by 2040). When asked what I mean, I generally say something like “the probability I would have if I had perfect knowledge and unlimited computation”—but that isn’t quite right, because if I had

*truly*perfect knowledge and unlimited computation I would be able to resolve the probability to either 0 or 1. Perfect knowledge and computation*within reason*, I guess? But that’s kind of hand-wavey. What I’ve actually been meaning is the butterfly probability, and I’m glad this concept/post now exists for me to reference!More generally I’d say it’s useful to make intuitive concepts more precise, even if it’s hard to actually use the definition, in the same way that I’m glad logical induction has been formalized despite being intractable. Also I’d say that this is an interesting concept, regardless of whether it’s useful :)

The Bayesian persuasion framework requires that the set of possible world states be defined in advance—and then the question becomes, given certain utility functions for the expert and decision-maker, what information about the world state should the expert commit to revealing?

I think that Bayesian persuasion might not be the right framework here, because we get to choose the AI’s reward function. Assume (as Bayesian persuasion does) that you’ve defined all possible world states.

^{[1]}Do you want to get the AI to reveal*all*the information—i.e. which particular world state we’re in—rather than a convenient subset (that it has precommitted to)? That seems straightforward: just penalize it really heavily if it refuses to tell you the world state.I think the much bigger challenge is getting the AI to tell you the world state

*truthfully*—but note that this is outside the scope of Bayesian persuasion, which assumes that the expert is constrained to the truth (and is deciding which parts of the truth they should commit to revealing).- ^
“World states” here need not mean the precise description of the world, atom by atom. If you only care about answering a particular question (“How much will Apple stock go up next week?” then you could define the set of world states to correspond to relevant considerations (e.g. the ordered tuple of random variables (how many iPhones Apple sold last quarter, how much time people are spending on their Macs, …)). Even so, I expect that defining the set of possible world states to be practically impossible in most cases.

- ^

# [Question] Three questions about mesa-optimizers

For personal reasons it made sense for me to calculate the percentage of Londoners who will have COVID this Thursday, the 16th. The number I got was much higher than I intuitively expected: 10%. Please point out any errors you see!

Among specimens collected in London 5 days ago, about 8000 were positive. This is relative to 4000 before the recent rise in cases, suggesting about 4000 are Omicron. Source

Omicron doubles at a rate of 2.5 days in the UK. Source

So among specimens collected Monday, we’d expect ~16k Omicron cases. Among specimens collected Thursday the 16th that should be ~35k.

As a ballpark guess, we might guess that about half of cases are caught, so that’s ~70k.

The typical time period between someone catching COVID and getting tested is 5 days. So the number of Londoners who will

*catch*COVID on Thursday is ~280k, since they’ll typically get tested 5 days (two doublings) after that. That’s about 3% of the population of London.Omicron grows by a factor of ~1.3 per day, so (3/1.3)% will catch COVID on Wednesday, and so on.

**The total percentage of Londoners who will****have****COVID on Thursday is thus ~10%**(summing the appropriate geometric series).

Thoughts?

I think we’re disagreeing on semantics. But I’d endorse both the statements “violence is bad” and “violence is sometimes good”.

I’m not sure. The strongest claim I make in that direction is that “Many in the rationalist sphere look down on tribalism and group identity.” I think this is true—I bet each of the people I named would endorse the statement “The world would be better off with a lot less tribalism.”

To be clear, I’m agreeing with Eliezer; I say so in the second paragraph. But for the most part my post doesn’t directly address Eliezer’s essay except in passing. Instead I point out: “Yeah, the ‘bad for reasoning about tribe affiliated subjects’ is a drawback, but here’s a benefit, at least for me.”

It’s true that I didn’t draw a distinction between tribalism and group identity. My reason for doing so was that I thought both terms applied to my three examples. I thought a bit about the distinction between the two in my mind but didn’t get very far. So I’m not sure whether the pattern I pointed out in my post is true of tribalism, or of group identity, or both. But since you pressed me, let me try to draw a distinction.

(This is an exercise for me in figuring out what I mean by these two notions; I’m not imposing these definitions on anyone.)

The word “tribalism” has a negative connotation. Why? I’d say because it draws out tendencies of tribe members to lose subjectivity and defend their tribe. (I was going to call this “irrational” behavior, but I’m not sure that’s right; it’s probably epistemically irrational but not necessarily instrumentally irrational.) So, maybe

*tribalism*can be defined as a mindset of membership in a group that causes the member to react defensively to external challenges, rather than treating those challenges objectively.(I know that I feel tribalism toward the rationalist community because of how I felt on the day that Scott Alexander took down Slate Star Codex, and when the New York Times article was published. I expect to feel similarly about EA, but haven’t had anything trigger that emotional state in me about it yet. I feel a smaller amount of tribalism toward neoliberalism.)

(Note that I’m avoiding defining

*tribes*, just*tribalism*, because what’s relevant to my post is how I feel about the groups I mentioned, not any property of the groups themselves. If you wanted to, you could define a tribe as a group where the average member feels tribalism toward the group, or something.)*Identity*is probably easier to define—I identify with a group if I consider myself a member of it. I’m not sure which of these two notions is most relevant for the sort of pattern I point out, though.

Good point! You might be interested in how I closed off an earlier draft of this post (which makes some points I didn’t make above, but which I think ended up having too high of a rhetoric to insight ratio):

“I don’t endorse tribalism in general, or think it’s a net positive. Tribalism strikes me as a symmetric weapon, equally wieldable by good and evil. This alone would make tribalism net neutral, but in fact tribalism corrupts, turning scouts into soldiers, making people defend their side irrespective of who’s right. And the more tribal a group becomes, the more fiercely they fight. Tribalism is a soldier of Moloch, the god of defecting in prisoner’s dilemmas.

This is somewhat in tension with my earlier claim that

*my*tribalism is a net positive. If I claim that my tribalism is net positive, but tribalism as a whole is net negative, then I’m saying that I’m special. But everyone feels special from the inside, so you’d be right to call me out for claiming that most people who feel that their tribalism is good are wrong, but I happen to be right. I would respond by saying that*among people who think carefully about tribalism*, many probably have a good relationship with it. I totally understand if you don’t buy that — or if you think that I haven’t thought carefully enough about my tribalism.But the other thing is, tribalism’s relationship with Moloch isn’t so straightforward. While on the inter-group level it breeds discord, within a tribe it fosters trust and cooperation. An American identity, and a British identity, and a Soviet identity helped fight the Nazis — just as my EA identity helps fight malaria.

So my advice on tribalism might be summarized thus: first, think carefully and critically about who the good guys are. And once you’ve done that — once you’ve joined them — a little tribalism can go a long way. Not a gallon of tribalism — beyond a certain point, sacrificing clear thinking for social cohesion becomes negative even if you’re on the good side — but a teaspoon.”

# Can group identity be a force for good?

Thanks for mentioning Asch’s conformity experiment—it’s a

*great*example of this sort of thing! I might come back and revise it a bit to mention the experiment.(Though here, interestingly, a participant’s action isn’t exactly based on the

*percentage*of people giving the wrong answer. It sounds like having one person give the right answer was enough to make people give the right answer, almost regardless of how many people gave the wrong answer. Nevertheless, it illustrates the point that other people’s behavior totally does influence most people’s behavior to quite a large degree, even in pretty unexpected settings.)

Hi! I just wanted to mention that I

reallyappreciate this sequence. I’ve been having lots of related thoughts, and it’s great to see a solid theoretical grounding for them. I find the notion that bargaining can happen across lots of different domains—different people or subagents, different states of the world, maybe different epistemic states—particularly useful. And this particular post presents the only argument for rejecting a VNM axiom I’ve ever found compelling. I think there’s a decent chance that this sequence will become really foundational to my thinking.