tlevin

One Year in DC

• tlevinMay 12, 2025, 4:05 AM

  50 points

  14

  on: tlevin’s Shortform

  Found this passage from Schelling’s Strategy of Conflict to be useful in understanding how norms react, and how they should react, to behavior that is not necessarily harmful but is often associated with harm, cf. “decoupling.” Emphasis mine:

  It is essential, therefore, for maximum credibility, to leave as little room as possible for judgment or discretion in carrying out the threat. If one is committed to punish a certain type of behavior when it reaches certain limits, but the limits are not carefully and objectively defined, the party threatened will realize that when the time comes to decide whether the threat must be enforced or not, his interest and that of the threatening party will coincide in an attempt to avoid the mutually unpleasant consequences.

  In order to make a threat precise, so that its terms are visible both to the threatened party and to any third parties whose reaction to the whole affair is of value to the adversaries, it may be necessary to introduce some arbitrary elements. The threat must involve overt acts rather than intentions; it must be attached to the visible deeds, not invisible ones; it may have to attach itself to certain ancillary actions that are of no consequence in themselves to the threatening party. It may, for example, have to put a penalty on the carrying of weapons rather than their use; on suspicious behavior rather than observed misdemeanors; on proximity to a crime rather than the crime itself. And, finally, the act of punishment must be one whose effect or influence is clearly discernible.

  In order that one be able to pledge his reputation behind a threat, there must be continuity between the present and subsequent issues that will arise. This need for continuity suggests a means of making the original threat more effective; if it can be decomposed into a series of consecutive smaller threats, there is an opportunity to demonstrate on the first few transgressions that the threat will be carried out on the rest. Even the first few become more plausible, since there is a more obvious incentive to fulfill them as a “lesson.” This principle is perhaps most relevant to acts that are inherently a matter of degree. In foreign aid programs the overt act of terminating assistance may be so obviously painful to both sides as not to be taken seriously by the recipient, but if each small misuse of funds is to be accompanied by a small reduction in assistance, never so large as to leave the recipient helpless nor to provoke a diplomatic breach, the willingness to carry it out will receive more credulity; or if it does not at first, a few lessons may be persuasive without too much damage.

  The threatening party may not, of course, be able to divide the act into steps. (Both the act to be deterred and the punishment must be divisible.) But the principle at least suggests the un-wisdom of defining aggression, or transgression, in terms of some critical degree or amount that will be deemed intolerable. When the act to be deterred is inherently a sequence of steps whose cumulative effect is what matters, a threat geared to the increments may be more credible than one that must be carried out either all at once or not at all when some particular point has been reached. It may even be impossible to define a “critical point” with sufficient clarity to be persuasive. To make the threatened acts divisible, the acts themselves may have to be modified. Parts of an act that cannot be decomposed may have to be left out; ancillary acts that go with the event, though of no interest in themselves, may be objects to which a threat can effectively be attached. For example, actions that are only preparatory to the main act, and by themselves do no damage, may be susceptible of chronological division and thus be effective objects of the threat. The man who would kick a dog should be threatened with modest punishment for each step toward the dog, even though his proximity is of no interest in itself.

• tlevinMay 12, 2025, 3:32 AM

  10 points

  5

  on: Con­sider not donat­ing un­der $100 to poli­ti­cal candidates

  Cross-posting this comment from the EA Forum:

  I basically agree with this (and might put the threshold higher than $100, probably much higher for people actively pursuing policy careers), with the following common exceptions:

  It seems pretty low-cost to donate to a candidate from Party X if...

  • You’ve already made donations to Party X. Larger and more recent ones are slightly worse, but as Daniel notes, even small ones from several elections ago can come back to bite.

  • You don’t see a realistic world where you go into the federal government during a Party Y administration even if you didn’t donate to Party X, because...

    • You don’t think you could go into the federal government at all (though as Daniel notes, you may not realize at the time of making the donation that you might want to later; what I have in mind is like, you have significantly below average people skills, and/​or you’ve somehow disqualified yourself).

    • You have a permanently discoverable digital paper trail of criticizing Party Y, e.g. social media posts, op-eds, etc.

    • You just don’t think you’d be able to stomach working in a Party Y administration. (Though consider asking, would you really not be able to stomach it for a few years if it seemed like an amazing career and impact opportunity?)

• tlevinApr 21, 2025, 4:14 AM

  3 points

  1

  in reply to: johnswentworth’s comment on: john­swent­worth’s Shortform

  I disagree that the default would’ve been that the board would’ve been “easy for the labs to capture” (indeed, among the most prominent and plausible criticisms of its structure was that it would overregulate in response to political pressure), and thus that it wouldn’t have changed deployment practices. I think the frontier companies were in a good position to evaluate this, and they decided to oppose the bill (and/​or support it conditional on sweeping changes, including the removal of the Frontier Model Division).

  Also, I’m confused when policy skeptics say things like “sure, it might slow down timelines by a factor of 2-3, big deal.” Having 2-3x as much time is indeed a big deal!

• tlevinApr 21, 2025, 3:46 AM

  8 points

  3

  on: tlevin’s Shortform

  Potential high-ROI purchase: lots of the same decent, cheap socks. I’ve saved several minutes per week of pairing socks by getting two packs of these. 12 pairs will set you back $30-$34, depending on size (plus shipping if applicable); they’re listed in Wirecutter’s “best no-show socks” (even though they do show while wearing most sneakers; aren’t all socks “no-show” socks if you posit a tall enough shoe?).

  (Of course, this is less useful if you’re fine with wearing mismatched socks anyway. But I think the asymmetry looks slightly bad and it sends a weakly bad signal re: conscientiousness, so I try to avoid it.)

• tlevinFeb 28, 2025, 3:23 AM

  1 point

  0

  in reply to: Anthony DiGiovanni’s comment on: tlevin’s Shortform

  Sequence thinking can totally generate that, but it seems like it is also prone to this kind of stylized simple model where you wind up with too few arrows in your causal graph and then inaccurately conclude that some parts are necessary and others aren’t helpful.

• tlevinFeb 25, 2025, 4:44 AM

  31 points

  3

  on: tlevin’s Shortform

  Biggest disagreement between the average worldview of people I met with at EAG and my own is something like “cluster thinking vs sequence thinking,” where people at EAG were often like “but even if we get this specific policy/​technical win, doesn’t it not matter unless you also have this other, harder thing?” and I was often more like, “Well, very possibly we won’t get that other, harder thing, but still seems really useful to get that specific policy/​technical win, here’s a story where we totally fail on that first thing and the second thing turns out to matter a ton!”

Skep­ti­cism to­wards claims about the views of pow­er­ful institutions

• tlevinDec 3, 2024, 4:31 PM

  3 points

  0

  in reply to: Zac Hatfield-Dodds’s comment on: A case for donat­ing to AI risk re­duc­tion (in­clud­ing if you work in AI)

  Agreed, I think people should apply a pretty strong penalty when evaluating a potential donation that has or worsens these dynamics. There are some donation opportunities that still have the “major donors won’t [fully] fund it” and “I’m advantaged to evaluate it as an AIS professional” without the “I’m personal friends with the recipient” weirdness, though—e.g. alignment approaches or policy research/​advocacy directions you find promising that Open Phil isn’t currently funding that would be executed thousands of miles away.

A case for donat­ing to AI risk re­duc­tion (in­clud­ing if you work in AI)

• tlevinNov 20, 2024, 8:22 PM

  5 points

  2

  in reply to: Orpheus16’s comment on: Akash’s Shortform

  Depends on the direction/​magnitude of the shift!

  I’m currently feeling very uncertain about the relative costs and benefits of centralization in general. I used to be more into the idea of a national project that centralized domestic projects and thus reduced domestic racing dynamics (and arguably better aligned incentives), but now I’m nervous about the secrecy that would likely entail, and think it’s less clear that a non-centralized situation inevitably leads to a decisive strategic advantage for the leading project. Which is to say, even under pretty optimistic assumptions about how much such a project invests in alignment, security, and benefit-sharing, I’m pretty uncertain that this would be good, and with more realistic assumptions I probably lean towards it being bad. But it super depends on the governance, the wider context, how a “Manhattan Project” would affect domestic companies and China’s policymaking, etc.

  (I think a great start would be not naming it after the Manhattan Project, though. It seems path dependent, and that’s not a great first step.)

• tlevinAug 30, 2024, 12:27 AM

  3 points

  2

  in reply to: habryka’s comment on: tlevin’s Shortform

  It’s not super clear whether from a racing perspective having an equal number of nukes is bad. I think it’s genuinely messy (and depends quite sensitively on how much actors are scared of losing vs. happy about winning vs. scared of racing).

  
  Importantly though, once you have several thousand nukes the strategic returns to more nukes drop pretty close to zero, regardless of how many your opponents have, while if you get the scary model’s weights and then don’t use them to push capabilities even more, your opponent maybe gets a huge strategic advantage over you. I think this is probably true, but the important thing is whether the actors think it might be true.

  In-general I think it’s very hard to predict whether people will overestimate or underestimate things. I agree that literally right now countries are probably underestimating it, but an overreaction in the future also wouldn’t surprise me very much (in the same way that COVID started with an underreaction, and then was followed by a massive overreaction).

  Yeah, good point.

• tlevinAug 29, 2024, 11:58 PM

  3 points

  0

  in reply to: habryka’s comment on: tlevin’s Shortform

  Yeah doing it again it works fine, but it was just creating a long list of empty bullet points (I also have this issue in GDocs sometimes)

• tlevinAug 29, 2024, 11:56 PM

  3 points

  2

  in reply to: habryka’s comment on: tlevin’s Shortform

  Gotcha. A few disanalogies though—the first two specifically relate to the model theft/​shared access point, the latter is true even if you had verifiable API access:

  1. Me verifying how many nukes you have doesn’t mean I suddenly have that many nukes, unlike AI model capabilities, though due to compute differences it does not mean we suddenly have the same time distance to superintelligence.

  2. Me having more nukes only weakly enables me to develop more nukes faster, unlike AI that can automate a lot of AI R&D.

  3. This model seems to assume you have an imprecise but accurate estimate of how many nukes I have, but companies will probably be underestimating the proximity of each other to superintelligence, for the same reason that they’re underestimating their own proximity to superintelligence, until it’s way more salient/​obvious.

• tlevinAug 29, 2024, 11:49 PM

  6 points

  1

  on: Monthly Roundup #21: Au­gust 2024

  In general, we should be wary of this sort of ‘make things worse in order to make things better.’ You are making all conversations of all sizes worse in order to override people’s decisions.

  Glad to be included in the roundup, but two issues here.

  First, it’s not about overriding people’s decisions; it’s a collective action problem. When the room is silent and there’s a single group of 8, I don’t actually face a choice of a 2- or 3-person conversation; it doesn’t exist! The music lowers the costs for people to split into smaller conversations, so the people who prefer those now have better choices, not worse.

  Second, this is a Simpson’s Paradox-related fallacy: you are indeed making all conversations more difficult, but in my model, smaller conversations are much better, so by making conversations of all sizes slightly to severely worse but moving the population to smaller conversations, you’re still improving the conversations on net.

• tlevinAug 29, 2024, 11:39 PM

  1 point

  0

  in reply to: habryka’s comment on: tlevin’s Shortform

  Also—I’m not sure I’m getting the thing where verifying that your competitor has a potentially pivotal model reduces racing?

• tlevinAug 29, 2024, 11:37 PM

  3 points

  0

  in reply to: habryka’s comment on: tlevin’s Shortform

  The “how do we know if this is the most powerful model” issue is one reason I’m excited by OpenMined, who I think are working on this among other features of external access tools

• tlevinAug 29, 2024, 11:35 PM

  3 points

  0

  in reply to: habryka’s comment on: tlevin’s Shortform

  If probability of misalignment is low, probability of human+AI coups (including e.g. countries invading each other) is high, and/​or there aren’t huge offense-dominant advantages to being somewhat ahead, you probably want more AGI projects, not fewer. And if you need a ton of compute to go from an AI that can do 99% of AI R&D tasks to an AI that can cause global catastrophe, then model theft is less of a factor. But the thing I’m worried about re: model theft is a scenario like this, which doesn’t seem that crazy:

  • Company/​country X has an AI agent that can do 99% [edit: let’s say “automate 90%”] of AI R&D tasks, call it Agent-GPT-7, and enough of a compute stock to have that train a significantly better Agent-GPT-8 in 4 months at full speed ahead, which can then train a basically superintelligent Agent-GPT-9 in another 4 months at full speed ahead. (Company/​country X doesn’t know the exact numbers, but their 80% CI is something like 2-8 months for each step; company/​country Y has less info, so their 80% CI is more like 1-16 months for each step.)

  • The weights for Agent-GPT-7 are available (legally or illegally) to company/​country Y, which is known to company/​country X.

  • Y has, say, a fifth of the compute. So each of those steps will take 20 months. Symmetrically, company/​country Y thinks it’ll take 10-40 months and company/​country X thinks it’s 5-80.

  • Once superintelligence is in sight like this, both company/​country X and Y become very scared of the other getting it first—in the country case, they are worried it will undermine nuclear deterrence, upend their political system, basically lead to getting taken over by the other. The relevant decisionmakers think this outcome is better than extinction, but maybe not by that much, whereas getting superintelligence before the other side is way better. In the company case, it’s a lot less intense, but they still would much rather get superintelligence than their arch-rival CEO.

  • So, X thinks they have anywhere from 5-80 months before Y has superintelligence, and Y thinks they have 1-16 months. So X and Y both think it’s easily possible, well within their 80% CI, that Y beats X.

  • X and Y have no reliable means of verifying a commitment like “we will spend half our compute on safety testing and alignment research.”

  • If these weights were not available, Y would have a similarly good system in 18 months, 80% CI 12-24.

  So, had the weights not been available to Y, X would be confident that it had 12 + 5 months to manage a capabilities explosion that would have happened in 8 months at full speed; it can spend >half of its compute on alignment/​safety/​etc, and it has 17 rather than 5 months of serial time to negotiate with Y, possibly develop some verification methods and credible mechanisms for benefit/​power-sharing, etc. If various transparency reforms have been implemented, such that the world is notified in ~real-time that this is happening, there would be enormous pressure to do so; I hope and think it will seem super illegitimate to pursue this kind of power without these kinds of commitments. I am much more worried about X not doing this and instead just trying to grab enormous amounts of power if they’re doing it all in secret.

  [Also: I just accidentally went back a page by command-open bracket in an attempt to get my text out of bullet format and briefly thought I lost this comment; thank you in your LW dev capacity for autosave draft text, but also it is weirdly hard to get out of bullets]

• tlevinAug 29, 2024, 9:43 PM

  16 points

  −9

  on: tlevin’s Shortform

  [reposting from Twitter, lightly edited/​reformatted] Sometimes I think the whole policy framework for reducing catastrophic risks from AI boils down to two core requirements—transparency and security—for models capable of dramatically accelerating R&D.

  If you have a model that could lead to general capabilities much stronger than human-level within, say, 12 months, by significantly improving subsequent training runs, the public and scientific community have a right to know this exists and to see at least a redacted safety case; and external researchers need to have some degree of red-teaming access. Probably various other forms of transparency would be useful too. It feels like this is a category of ask that should unite the “safety,” “ethics,” and “accelerationist” communities?

  And the flip side is that it’s very important that a model capable of kicking off that kind of dash to superhuman capabilities not get stolen/​exfiltrated, such that you don’t wind up with multiple actors facing enormous competitive tradeoffs to rush through this process.

  These have some tradeoffs, especially as you approach AGI—e.g. if you develop a system that can do 99% of foundation model training tasks and your security is terrible you do have some good reasons not to immediately announce it—but not if we make progress on either of these before then, IMO. What the Pareto Frontier of transparency and security looks like, and where we should land on that curve, seems like a very important research agenda.

  If you’re interested in moving the ball forward on either of these, my colleagues and I would love to see your proposal and might fund you to work on it!

• tlevinJul 31, 2024, 7:10 AM

  3 points

  2

  in reply to: habryka’s comment on: tlevin’s Shortform

  Seems cheap to get the info value, especially for quieter music? Can be expensive to set up a multi-room sound system, but it’s probably most valuable in the room that is largest/​most prone to large group formation, so maybe worth experimenting with a speaker playing some instrumental jazz or something. I do think the architecture does a fair bit of work already.