The median researcher hypothesis seems false. Something like an 80⁄20 distribution seems much more plausible, and is presumably more like what you’d find for measurable proxies of ‘influence on a field’ like number of publications in “top tier” journals, or number of researchers in the field who were your grad student. Voting “no”.
Unnamed
Karma: 8,112
Mediocre criticism can get plenty of upvotes as long as it’s a culture fit.
If the author does a good job of pitching it to Less Wrongers, then the critical post can activate readers’ it’s virtuous to be open-minded mindset and turn their critical faculties towards the thing that the post is criticizing and away from the post itself. So instead of evaluating the post according to their ordinary standards of epistemics and quality, they instead try to find anything in it that seems good / insightful / overly neglected / provoking of new useful thoughts / on a promising track.
Seems like you’re mushing together several loosely related things, including what we might call model-based motivation, explicit long-term planning, unified purpose, and precisely targeted goals.
Model-based motivation: being motivated to do something in a way that relies on your internal models of the world, not just on direct sensory rewards.
Explicit long-term planning: being aware of your goal, explicitly planning ways to achieve it, following those plans including over periods of months or years.
Unified purpose: a person’s motivations and actions in a domain fitting together coherently to work towards a single purpose, even across contexts.
Precisely targeted goals: having the goal precisely match something that can be specified on other grounds besides what we can empirically observe that people aim for (like “inclusive genetic fitness” which is picked out by theory).
The godshatter post is mainly about the last two—people have a collection of fragmented motivations which helped towards the selected-for purpose in the contexts where we evolved. Your argument here is mainly about the first two.
I think that the first two are pretty common, and are found in human romantic/reproductive goals, e.g. long-term planning around having kids, or motivations to improve ones appearance in ways that you expect potential partners to find attractive. I think that the last two are pretty rare, including for status—most people have a collection of somewhat-status-related motivations (though perhaps a small fraction of people (sociopaths?) have status as a more unified goal), and I haven’t seen anyone specify the “status” target well enough to even check if people’s motivations aim at that precise target.
You seem to think that this post poses a single clear puzzle, of the sort that could have a single answer.
I disagree. I think the post has clarity problems (especially in its definition of poverty in terms of “desperate scrabbling”, which conflates a lack of at least one essential material resource with anything at all that a person might desperately care about) and kind of gestures at various questions related to poverty.
I haven’t given that article a close read, but on a quick look through it I find it basically not at all compelling.
It looks like it’s the genre of ‘this part of reality is surprisingly detailed, therefore be paranoid/nihilistic/cynical about it’.
This genre of article is saying: there’s this concept that you’ve been using, which you’ve been treating as a clean abstraction without really thinking much about where it can from, but if you look at where it comes from, there is a bunch of messy detail & judgments calls.
And that much, often, is true. But it’s written with an air of suspicion, or with explicit claims that therefore it’s all just a bunch of made-up nonsense. Which does not follow.
Numbers that involve judgment calls aren’t in general fake/nonsense/bullshit. I regularly use subjective probabilities, Fermi estimates, etc., and I imagine that you do too.
If you want, you can have the takeaway from this sort of article: “that’s right, I’ve been using this concept without really understanding the messy detail behind it. Do I care enough about this to want to understand where it comes from?”
If so, then you can go try to learn about it, using the processes that you usually use to learn about things. Try to find writing by economists explaining where inflation numbers and “real GDP” numbers come from, or some narrower question that you could dig into enough so that you have a . Have a conversation with Claude about it. Make a google doc where you think through what you would do if it was up to you to come up with number for “real GDP”, or to do whatever tasks people do when they rely on “real GDP” numbers. Etc.
This post of yours looks like it’s kinda trying to do some of that, although the things that you’d want to learn about might not fit into a lesswrong comment, and I don’t know if you’ll find anyone who is sufficiently well-informed about real GDP calculations and willing to spend the time to leave a detailed comment for you to get a good object-level answer here. And it reads like you’ve already maybe 70% bought in to the mood & the narratives of the post you’ve linked, which is not something that I’d recommend with this genre of post before you’ve tried to learn about it elsewhere. Maybe you can come back to this post afterwards and consider its reasoning after you have more grounding in the topic, but relying on this person’s judgment & narratives about some topic just because he’s the one who pointed out to you that it is surprisingly detailed seems like bad process.
To get a little more object-level: One thing that’s missing from the article (and your takeaways from it) is that most of the judgment calls that the economists who come up with “real GDP” numbers make are at the process level, of what procedures to use to assign a number to real GDP for a country for a particular year, given the various complications. When trying to answer a question like “what was US GDP in 1946“ they have limited degrees of freedom because they’re mostly just relying on the processes that they’ve decided to apply to answering that sort of question for all countries & years. Which is pretty different from “BEA economists eye-balling a bunch of factors and coming up with a number that “seems right””, even if both involve judgment calls.
Did you have a different vision for how to get really good AI X-risk legislation passed?
I’d interpreted your post has already implicitly sharing something like orthonomal’s view, since I took you to be arguing that we should prioritize getting a small number of legislators who really Get It.
It sounds like your view is that (say) a House with 5 legislators who are amazing on AI X-risk, 15 who seem like they’re kinda pretty good, and 415 others is actively worse than one with 5 amazing legislators and 430 others?
I’m not sure why you think this. I’d think that most of the ways in which the pretty good legislators could be disappointing would make them more similar to the 415 others, or less influential, rather than actively worse. And often it would still be somewhat helpful to have them in Congress, e.g. they’d generally be more likely than random legislators to vote for a good AI bill that has a chance at becoming law.
One big way it could backfire to have a pretty-good-seeming legislator in the house is if they become a leading voice on AI while having misguided views on AI. But the concern about candidates who have a combination of prioritizing AI, being very competent, and having misguided views on AI feels different than just having extremely high standards for amazingness on AI X-risk.
The first is simple, unemployment. It’s calculated in a way that is very favorable to the government[1], because the government decides how it’s calculated and generally wants to look like things are going well. Labor force participation, a statistic that more accurately captures the share of the productive population that is being squandered, has fallen precipitously from 2005 to around 2015, enjoyed a slight increase from 2015 to 2019, and then taken a nosedive afterwards, never recovering to its 2019 high. Since 2005, a full four percent of the population—one out of 25 people—have dropped out of the labor force. This is the sort of thing that affects everything, from the national psyche to the social fabric to, of course, our ability to use the country’s human resources efficiently.
Prime age employment to population ratio is a better measure, and it does not show a decline since 2005.
The measure that you picked goes down if the population gets older and includes a larger share of retired people (which it has) or if more people age 16-24 are in school rather than working (which has also been happening).
Rows 1677, 54048, 93530, and 141774 look anomalous—they should be guaranteed wins but are marked as losses. (3 of them pit 1 Laser Lance & 1 Rail Rifle vs. 2 Arachnoid Abominations.)
Criterion of rightness vs. decision procedure (also: multi-level utilitarianism)
Ideas similar to these were present to some degree among early utilitarians like Mill and Sidgwick, and the concepts were crystallized by later philosophers including Bales (1971) and Hare (1981).
Updated choice
I’ll go with ABDMOPV.
My second choice would be FGHOPTV.
My process
This is mainly based on a linear model which includes all pairwise interactions, with a few adjustments:
also accounts for number of sweet dishes (AEGH) - too many or too few is bad
also accounts for number of spicy dishes (CFKSV) - too many or too few is bad
also accounts for total number of dishes—too many or too few is bad
I made the linear interaction sparse, rounding small coefficients to zero
Displacer Dumplings seem to be irrelevant to meal quality, so I’m averaging in the same meal with or without D (but picking whichever one happened to do better)
I slightly averaged in a couple other models that I tried which aren’t as accurate on their own as the linear interaction model
I haven’t been able to reduce the error all the way down to where it seems to be among repeat meals, but it’s not that far off.
My opinion on other entries
I would attend James Camacho’s AGORTV feast if I wasn’t hosting my own
A big part of what Evrart is doing is frame setting. What kind of person is he, what is the nature of your relationship with him and how does that relate to specific things that you might do, what is his role in the city, etc.? He comes right out and tells you, trying to directly influence how you think of him and the social scripts & roles that you see as applying. And if you come to him in another frame that he doesn’t want, he sidesteps that framing rather than interacting in the role that that frame puts him in.
Also related: his larger-than-life personality. He crafts a personality that 1) fits with the frames he wants (friendly, gregarious, nice) and which 2) gives him leeway for acting outside normal social expectations, such as by putting forward a frame or sidestepping one (it’s a strange thing to do but that’s Evrart being Evrart). Something about his personality even makes it seem kind of okay for him to put forward frames that seem implausible or inaccurate. (Though IMO he is not able to do this in a way that avoids seeming fishy.)
Updated observations
Taking another look after sleeping on it, I did find some of the sorts of meal-crafting patterns I was expecting, with effects from number of sweet dishes (AEGH) and number of spicy dishes (CFKSV). I actually crafted a ‘number of sweet things’ variable last night (AEGHP), but I just threw it in a linear model rather than looking at the data.
Updated choices
CDGHMOPS is still the best option according to my new best model. ABDMOPTV the new choice for doing well according to every decent model while still being close to the best according to my new best model. If I was picking now I’d go with the latter.
If I was going to replicate a successful historical meal, it would be
#1078, ABFGKOP, which had a quality of 18 and models well. That’s a safe option for a good feast.
But I’d rather do something original, and
the best option according to the best model I’ve found is CDGHMOPS. Another plausible choice is AGOPRTV, which does very well in every decent model that I’ve tried.
Observations
With American Thanksgiving, I’d mostly expect the more dishes the better, because each person can choose which subset of the dishes they want to eat. The biggest costs are to lack of variety, because each person wants variety (e.g. some protein and some dessert) and different people want different things (e.g. spicy, not spicy, meat, meatless). Also, occasionally dishes are complementary (e.g. pie & ice cream, or cake & ice cream).
The Feasts here are apparently not like that, because that is not how the data looks. Perhaps the dishes have strong odors, or the custom is to have some of everything, or the foods have unexpected magical interactions in your belly.
or “7 Vicious Virtues”
It’s interesting that this post is framed in terms of status. That seems more like an illustration of the flexibility of what “status” can mean than something essential to the main point of the post.
I could imagine pretty much the same point being made, with most of the same content, without referencing status at all (for a big success don’t just join the default competitive arena). Or it could instead be framed as how to be high status, where big success option is the high-status option (climbing the ordinary competitive ladder is upper-middle class at best, actual high status routes around that whole competition).
Within the compressed summary “Status Is The Game Of The Losers’ Bracket”, a lot of the post’s main content winds up within the particular way that the word “status” is being used here.
I count 5 strategies in this post & the previous one, rather than 3:
Blinding. Block information input from the adversary to you.
Privacy. Block information output from you to the adversary.
Disempowerment. Don’t let the adversary have control over parts of the environment that you care about.
Vindictiveness. Do things that are opposed to the adversary’s interests.
Randomness. Do things that are hard for the adversary to predict.
#3 Disempowerment was least explicitly stated in your writing but was present in how you talked about purging / removal from your environment. Examples: Don’t have a joint bank account with them, don’t appoint them to a be in charge of a department in your organization, don’t make agreements with them where they have the official legal rights but there’s a handshake deal that they’ll share things with you.
Truman’s response to the Red Scare included all (or at least most) of the first 4 strategies. It was primarily #2 Privacy—in fact the Soviet spies were mainly doing espionage—acquiring confidential information from the US government—and purging them was blocking them from getting that information. But Truman was worried about them doing subversion (getting the US government to make bad decisions) which would make purging them #3 Disempowerment. And executing them (rather than just firing them) makes it #4 Vindictive too.
The Madman Theory example in the other post is mainly about vindictiveness (it’s a threat to retaliate), even though it’s done in a way that involves some randomness.
#5 Randomness feels least like a single coherent thing out of these 5. I’d break it into:
5a Maximin. Do things that work out best in the worst case scenario. This often involves a mixed strategy where you randomize across multiple possible actions (assuming you have a hidden source of randomness).
5b Erraticness. Thwart their expectations. Don’t do the thing that they’re expecting you to do, or do something that they wouldn’t have expected.
Though #5b Erraticness seems like an actively bad idea if you have been fully diagonalized, since in case you won’t actually succeed at thwarting their expectations and your erratic action will instead be just what they wanted you to do. It is instead a strategy for cat-and-mouse games where they can partially model you but you can still hope to outsmart them.
If you have been diagonalized, it’s better to limit your repertoire of actions. Choose inaction where possible, stick to protocol, don’t do things that are out of distribution. The smaller the set of actions that you ever do, the fewer options the diagonalizer has for what to get you to do. A hacker gets a computer system into a weird edge case, a social engineer gets someone to break protocol, a jailbreaker gets an LLM into an out-of-distribution state. An aspiring diagonalizer also wants to influence the process that you use to make decisions, and falling back on a pre-existing protocol can block that influence. I would include this on my list of strategies, maybe #6 Act Conservatively.
Looking back through these, most of them aren’t that specific to diagonalization scenarios. Strategies 4 (Vindictiveness) & 5a (Maximin) are standard game theory which come up in lots of contexts. I think that strategies 1-3 fall out of a fairly broad sense of what it means for someone to be an adversary—they are acting contrary to your interests, in a way that’s entangled with you; they’re not just off somewhere else doing things you don’t like, they are in some way using you to get more of the thing that’s bad for you. In what ways might they be using you to get more of the thing? Maybe they’re getting information from you which they can then use for their purposes, maybe they’re trying to influence what you do so you do what they want, maybe you’ve let them have control over something which you could have disallowed. Strategies 1 (Blinding), 2 (Privacy), and 3 (Disempowerment) just involve undoing/blocking one of those.
That helped give me a better sense of where you’re coming from, and more of an impression of what the core thing is that you’re trying to talk about. Especially helpful were the diagonalization model at the end (which I see you have now made into a separate post) and the part about “paranoia to me is centrally invoked by high-bandwidth environments that are hard to escape from” (while gesturing at a few examples, including you at CEA). Also your exchange elsewhere in the comments with Richard.
I still disagree with a lot of what you have to say, and agree with most of my original bullet points (though I’d make some modifications to #2 on your division into three strategies and #6 on selective skepticism). Not sure what the most productive direction is to go from here. I have some temptation to get into a big disagreement covid, where I think I have pretty different models than you do, but that feels like it’s mainly a tangent. Let me instead try to give my own take on the central thing:
The central topic is situations where an adversary may have compromised some of your internal processes. Especially when it’s not straightforward to identify what they’ve compromised, fix your processes, or remove their influence. There’s a more theoretical angle on this which focuses on what are good strategies to use in response to these sorts of situations, potentially even what’s the optimal response to a sufficiently well-specified version of this kind of scenario. And there’s a more empirical angle which focuses on what do people in fact do when they think they might be in this sort of situation, which could include major errors (including errors in identifying what situation you’re in, e.g. how much access/influence/capability the adversary has in relation to you or how adversarial the relationship is) though probably often involves responses that are at least somewhat appropriate.
This is narrower than what you initially described in this post (being in an environment with competent adversaries) but broader than diagonalization (which is the extreme case of having your internal processes compromised, where you are fully pwned). Though possibly this is still too broad, since it seems like you have something more specific in mind (but I don’t think that narrowing the topic to full diagonalization captures what you’re going for).
I don’t think it’s specific to sensitive topics, Richard just does a lot of sloppy thinking when he tries to engage with politics. His post/talk on more mundane political topics also led to a lot of people on LW & the EA Forum pointing out things he got wrong.