Arjun Panickssery

• Arjun Panickssery 26 Jul 2026 1:53 UTC

  35 points

  18

  in reply to: Thomas Kwa’s comment on: nikola’s Shortform

  Before I started at METR I dropped out of Caltech and had written two random interp papers

  Not to belabor your specific case—I agree directionally that there’s no intense gatekeeping or connections needed—but you worked at Jane Street and MIRI and then did MATS before joining METR right . . . OP’s claim isn’t that specific time-consuming credentials are needed but that even “smarts + unpredictable fit” things are hard to signal in an application from outside (without PhDs or work experience).

Post­ing Some Prompts

Just Post­ing Some Prompts

• Arjun Panickssery 6 Jul 2026 3:17 UTC

  14 points

  0

  on: Ar­jun Pan­ickssery’s Shortform

  Hard to make a list off the top of my head—maybe I could keep a running list and post it when it reaches a certain length—but I feel like the LessWrong react options are often disappointing. For example, I might search for a react that means “can you define this /​ be more specific /​ what does this mean” but fail to find one.

  Meanwhile, there are a bunch of reacts whose purpose I don’t understand: Moloch, Paperclip, “I beseech you!”

  I also wouldn’t use some of the complimentary ones, like “Concrete” or “Scout Mindset,” because I imagine they would seem patronizing, pointless, etc. in any context.

  My intuition for reacts is that they compactly add info that would be cluttering as a reply to a lengthy post. For example:

  1. If a lengthy post contains “I have often heard AI safety researchers say they are motivated by high earnings” then I could react with “+1″ to indicate that I have also heard that, instead of saying so in a very short reply

  2. If a post has a typo, I could react with “Typo” (though I’ve never actually done that)

  3. The “X% Likely” reacts also fit into this theme.

  In particular, I imagine using these when I’m not part of the thread (because otherwise I could mention them in the reply), which means that it’s not clear how the writer is supposed to engage, so the audience is mainly other readers.

• Arjun Panickssery 30 Jun 2026 16:14 UTC

  4 points

  0

  in reply to: lc’s comment on: lc’s Shortform

  There is a common impulse to irrationally favor the apparently weaker party in any (even consensual and transparent) transaction. For example, many people dislike when a low-skill worker agrees to work for low wages, or a tenant agrees to rent with the possibility of high rent increases, or when a gambler agrees to partake in a casino that includes predictable losses and a policy of banning winning players. This is just another example of that. In reality it makes perfect sense to take pleas, or settlements, etc.

• Arjun Panickssery 30 Jun 2026 3:24 UTC

  4 points

  0

  in reply to: leogao’s comment on: leogao’s Shortform

  Presumably varies a lot by domain. (Or equivalently, maybe in some domains the “do something you believe in” advice will lead you to rationally do legible stuff like accumulate credentials, because other paths will be transparently ineffective.)

• Arjun Panickssery 28 Jun 2026 1:17 UTC

  4 points

  2

  in reply to: Linch’s comment on: Linch’s Shortform

  I mean at home.

  That’s interesting. I guess I’ve always gotten an email saying “time to check in” or whatever, and gone to the link, and entered the confirmation code like AUWKDS or whatever, and gotten my ticket. Maybe the only exception was when I was abroad and the airline/​airport required in-person check-in for some visa-related reason.

• Arjun Panickssery 28 Jun 2026 0:50 UTC

  4 points

  0

  in reply to: Linch’s comment on: Linch’s Shortform

  How could this be? You can just capture a screenshot of the boarding pass right

• Arjun Panickssery 24 Jun 2026 3:12 UTC

  12 points

  2

  in reply to: LawrenceC’s comment on: LawrenceC’s Shortform

  manage to overcome the millions of dollar spent against him by Leading the Future

  Reacted with “thumbs down” because this misleads the reader into thinking that OAI-associated PACs spent more than Anthropic-associated PACs, when the opposite is true.

  Then I got to the part of your post when you said so yourself, but I don’t know how to remove reacts in any case.

• Arjun Panickssery 5 Jun 2026 0:39 UTC

  57 points

  21

  on: Ar­jun Pan­ickssery’s Shortform

  Bertrand Russell’s parents died by the time he was four years old and in 1876 he went to live with his grandfather, the last Whig prime minister, who as a young man had met Napoleon at Elba.

  In 1966 at the age of 94 he met Paul McCartney and converted him to his anti-war stance on Vietnam.

  Previously, I had figured that this lifespan (roughly 1870 to 1960) was the most extreme length of history for someone to live through. You have to be born early enough to remember a time when big cities didn’t have street lights or automobiles, while ideally living to see the atom bomb and first man in space (1961).

  For example:

  • Churchill (1874–1965) rode in Britain’s last great cavalry charge (Omdurman 1898) and later in life ordered Britain’s first hydrogen-bomb program.

  • W.E.B. DuBois (1868–1963) was born three years after the Civil War but died before the March on Washington.

  Other such lifespans were Laura Ingalls Wilder (1867–1957), Frank Lloyd Wright (1867–1959), W. Somerset Maugham (1874–1965), Picasso (1881–1973).

  On Twitter people pointed outside the West, the same lifespan was even crazier, if they were born into totally pre-industrial worlds e.g.

  • “Syngman Rhee (1875-1965) who was one of the last people in Korea to pass the old Confucian civil service examinations and later became the President of South Korea, playing a pivotal role in the early Cold War … He was born in a preindustrial world where the old Confucian social order still held sway and nobody around him had ever seen a train and died in a world of jet planes and manned space travel.” @avrilbradley23

  • “A curious Surmic born then would have seen the forging of the Ethiopian Empire, the end of the witch-chiefs, Christian missionaries, the Italian conquest, literacy, new crops, and maybe the moon landing.” @Peter_Nimitz

  Meanwhile I had figured I was going to live through the “Great Stagnation” and the world would gradually become a humongous nursing home. But it seems like AGI is likely to keep the show going.

• Arjun Panickssery 31 May 2026 20:05 UTC

  2 points

  0

  in reply to: Arjun Panickssery’s comment on: Linch’s Shortform

  Also I think “put on” means “affected” and “worn out” means “anachronistic + a specific narrative of why it’s anachronistic, i.e. because the advice became worn out.”

• Arjun Panickssery 31 May 2026 20:01 UTC

  4 points

  0

  in reply to: Linch’s comment on: Linch’s Shortform

  (I didn’t disagree-vote).

  I considered “Latin,” which obviously comes from Latin, but via Old English so I think it’s fine. (Note that “Anglo” also comes from Latin, so a better case is that I should have just said “English.”) I don’t think it’s necessary to come up with made-up words for other places or languages that have their own names for this purpose.

  I agree that “plain” and “appear” come from Old French and those were oversights. And it shows that it’s hard to actually write this way without oversights.

• Arjun Panickssery 31 May 2026 6:53 UTC

  6 points

  0

  in reply to: yams’s comment on: Linch’s Shortform

  Quick search indicates that Churchill is the originator of the sentiment

  Where did you hear that? The Fowlers’ classic 1906 guide says “Prefer the Saxon word to the Romance” and the opinion was already standard at that time.

• Arjun Panickssery 31 May 2026 6:48 UTC

  2 points

  −4

  in reply to: Linch’s comment on: Linch’s Shortform

  Did you use Claude for the rewrite? I think you can do a much less weird one (and what is “Walnut Folk”?) e.g.

  I think that “to talk with folks plainly, use words with Anglo-Saxon rather than Latin roots” is itself put on and worn out.

  I think most good writers already have a good “feel” for which words are mainstream or over-the-top. But if you can’t tell, it’s better to look at how often the word appears rather than its roots.

  And indeed this is a plainer style.

• Arjun Panickssery 31 May 2026 1:11 UTC

  3 points

  1

  in reply to: cubefox’s comment on: Ar­jun Pan­ickssery’s Shortform

  This is similar to @habryka’s reply here where I agree with the statements in the reply but I don’t think they respond to my objection.

  If I understand your two points correctly they are that

  1. An open-question critique of the idealization-procedure definition can be applied to any conceptual analysis. Yes, sure. (Irrelevant but I also don’t think the analysis of concepts is very useful.)

  2. There is no is-ought gap because an “ought sentence” can be rephrased as an “is sentence.”

  But these only address a weak “semantic” interpretation of my objection to the analysis when what I am questioning is why the proposed analysis produces normative authority. My complaint isn’t the general complaint that to define the good as the product of an idealization procedure is either trivial or false, but that there’s this actual thing (normative authority) that isn’t addressed. Likewise with (2), you can certainly rephrase an “ought sentence” into an “is sentence” but that doesn’t change it from a normative to a descriptive claim.

  My question is about how an idealization procedure (like extrapolated volition or whatever else) can actually have moral authority if the whole procedure is specified in non-normative terms.

• Arjun Panickssery 31 May 2026 0:54 UTC

  3 points

  1

  in reply to: habryka’s comment on: Ar­jun Pan­ickssery’s Shortform

  I agree with everything in this response. In particular, I don’t mean to “deny the ability to do some amount of logical inference on top of your preferences.”

  My point is that it doesn’t answer the key metaethical question of why you ought to act according to any of those ideas.

• Arjun Panickssery 30 May 2026 19:48 UTC

  3 points

  1

  in reply to: hypnosifl’s comment on: Ar­jun Pan­ickssery’s Shortform

  For believers in scientific reductionism, moral realism based on a priori knowledge or fixed “human nature” or mental access to a realm of platonic moral truths is not plausible

  Sure, but this is a case for nihilism or similar views.

  In the moral realm, it may be that a combination of memetic and biological evolution would tend to cause strong convergence on certain norms in the long term

  Sure, but

  1. this doesn’t explain where the moral authority comes from, i.e. why you ought to follow the principles that could result from this process

  2. in particular, the specific “evolutionary” formula invites an evolutionary debunking, because the theory of natural selection suggests that we converge on moral principles that tend to produce persistent societies or genes or similar, rather than ones which are morally good

  3. a few of your points reference the “growth of knowledge” or “growth of reason or rationality” but I don’t see why (1) the described idealization procedure points toward those things or (2) why those things are good

• Arjun Panickssery 30 May 2026 4:43 UTC

  3 points

  1

  in reply to: cubefox’s comment on: Ar­jun Pan­ickssery’s Shortform

  This doesn’t really address my objection but just labels it.

  If I understand correctly, Yudkowsky merely asserts that real moral knowledge is found by

  running a certain logical function over possible states of the world, where this function is analytically identical to the result of extrapolating our current decision-making process in directions such as “What if I knew more?”, “What if I had time to consider more arguments (so long as the arguments weren’t hacking my brain)?”, or “What if I understood myself better and had more self-control?”

  But this is an idealization procedure, and so it falls into my dichotomy:

  In particular all of the theories based on an idealization procedure fail because either

  1. The idealization procedure is taken to include moral knowledge, creating circularity, or

  2. The idealization procedure only includes rationality in the making of non-moral judgments, knowledge of non-moral facts, etc, in which case this is a reductionist meta-ethics that doesn’t actually cross the is-ought gap (i.e. it remains an open question whether the idealized attitudes would be good).

  I don’t see a clear moral/​evaluative claim baked into the listed examples there, so therefore it maintains the problem of explaining why the outcome of the idealization procedure is actually good and why you ought to care about it, i.e. crossing the is-ought gap.

  (My objection is similar or maybe the same to the open-question objection to analytic naturalism, of which analytic functionalism is one type.)

• Arjun Panickssery 30 May 2026 4:31 UTC

  6 points

  1

  in reply to: habryka’s comment on: Ar­jun Pan­ickssery’s Shortform

  No, it just doesn’t assume that. It’s totally fine for different people to want different things, and for their extrapolated values to diverge, under Eliezer’s metaethics.

  Ah ok. I admit I don’t know much about CEV, compared to the other two listed items in my top-level post. This document admits (emphasis):

  Q9. How does the dynamic force individual volitions to cohere? (Frequently Asked)

  The dynamic doesn’t force anything. The engineering goal is to ask what humankind “wants,” or rather what we would decide if we knew more, thought faster, were more the people we wished we were, had grown up farther together, etc. “There is nothing which humanity can be said to ‘want’ in this sense” is a possible answer to this question. Meaning, you took your best shot at asking what humanity wanted, and humanity didn’t want anything coherent.

  It defines coherence as “Strong agreement between many extrapolated individual volitions which are unmuddled and unspread in the domain of agreement, and not countered by strong disagreement.” So while it is conceded, in passing, that there might not be a result, I assumed that Yudkowsky thinks it’s plausible, because otherwise it wouldn’t make sense to advocate for CEV as the target for AI alignment. (I guess it’s possible that he concluded that it would be better for an AI not to do anything in that case, as a safe failure mode, versus to act on a different alignment target.)

  ---

  Clearly there are some actions you can take that make you think you will make better ethical judgements in the future.

  This doesn’t address the is-ought gap. I agree that if you already accept moral realism then this kind of thing is a relevant consideration, but positing an idealization procedure doesn’t solve meta-ethics. Things like “sleeping enough” only corrects non-moral defects like fatigue but doesn’t address the question of whether the resulting judgments are objectively good. In contrast, the “be more the people you wished you were” in Yudkowsky’s idealization procedure introduces moral knowledge and values (insofar as “wishing” is an evaluative attitude), but that creates circularity.

  In particular all of the theories based on an idealization procedure fail because either

  1. The idealization procedure is taken to include moral knowledge, creating circularity, or

  2. The idealization procedure only includes rationality in the making of non-moral judgments, knowledge of non-moral facts, etc, in which case this is a reductionist meta-ethics that doesn’t actually cross the is-ought gap (i.e. it remains an open question whether the idealized attitudes would be good).

  My broader accusation is that this kind of talk is used for crypto-realism; people want to basically talk in terms of stance-independent moral facts. But they merely frame the discussion in terms of what their idealized self would believe, when in reality the idealization procedure is either circular or can’t cross the is-ought gap and introduce moral knowledge. You yourself just talked in terms of “progress on ethics” and “better ethical judgments” but by that you could mean either

  1. “Progress on figuring out what my idealized self would think /​ what judgments he would make”—how does this illuminate any metaethics?

  2. “Progress on figuring out objective, or stance-independent, ethics/​judgments”—how would the idealized self be authoritative about that, especially if they diverge among people?

• Arjun Panickssery 29 May 2026 22:04 UTC

  7 points

  −3

  in reply to: Mitchell_Porter’s comment on: Ar­jun Pan­ickssery’s Shortform

  But what is the value of these “dispositions”? I certainly have some dispositions; for example, my intuition is that Mt. Kilimanjaro is more beautiful than a random pile of garbage.

  This “extrapolation” concept assumes a bunch of stuff:

  1. That if everyone underwent an idealization procedure, they would find some kind of common ground

  2. That people should care, personally, about what their idealization procedure would produce

  My point is that no “idealization procedure” solves the hard problem, which is crossing the is-ought gap i.e. going from facts about the world or about your impressions and deriving moral principles.