davekasten

Karma: 1,777

davekasten 7 Jan 2026 22:04 UTC
43 points
20
on: davekasten’s Shortform
Something I believe:
The reason that society isn’t currently freaking out about AI taking artists’ jobs is mainly that we’ve historically thought of artists’ jobs as inherently precarious, and so a new report of them being precarious for new reasons doesn’t surprise anyone. The moment it takes a “so stable your mom wants you to study it in school” job, that all will change for the stable folks, but not the artists, unfortunately. After all, they’re just artists...
(If you say that this means that I don’t care about artists or their financial challenges, you’re wrong. This sucks, I’m predicting, not endorsing, a likely scenario.)

davekasten 23 Dec 2025 17:00 UTC
6 points
0
in reply to: MondSemmel’s comment on: Elizabeth’s Shortform
Yup, this is very much part of it. But overall, it’s the tightening of a thousand screws rather than One Weird Trick.
In the House, in addition to the tradition changes that MondSemmel mentions, successive speakers over time have modified the formal House Rules iteratively to (generally) consolidate more power under the Speaker in a bunch of bureaucratically technical but important ways. (E.G., the Federalist Society on the right argues that Gingrich’s decision to cut committee staff sizes and impose term limits for committee chairs nerfed the power of committees relative to the Speaker https://fedsoc.org/commentary/fedsoc-blog/navigating-the-rules-of-the-people-s-house). Note that some view the current Speaker as intentionally choosing to begin to reverse this trend as part of commitments he voluntarily made to his side of the aisle to somewhat empower individual Members, but this is seen even by his biggest fans as only a first step, and certainly not universally agreed upon even within his own Party.
In the Senate, similar trends are combined with a slow but fairly steady erosion of the scope of the filibuster (as well as the maximization of procedural hacks around the filibuster, such as how to use reconciliation), which de facto increases the power of the Senate Majority Leader in closely-divided Senates (most have been for the past 20 years). But this is not as far along as in the House, and various Senate Majority Leaders have approached this question differently, even within the same party, so bright line conclusions are harder to draw here. (E.G., Reid and Schumer had somewhat different approaches from each other, as do McConnell and Thune)

davekasten 22 Dec 2025 15:57 UTC
5 points
0
in reply to: Raemon’s comment on: Elizabeth’s Shortform
This is a good summary by Vox, which in turn points to a bunch of deeper writeups if helpful https://archive.is/o6qj8

davekasten 22 Dec 2025 4:53 UTC
25 points
3
in reply to: Elizabeth’s comment on: Elizabeth’s Shortform
Congressmembers do have power, but it’s...weirdly distributed at the veto points? There is some horse trading, but it’s maximally in the “you gotta trust me it’s there” black-box aspects of the system (e.g., the famously opaque negotiation inside the NDAA process).

But power has definitely drained away from individual members; the standard account is roughly that a combo of 0) Increased transparency into Congress via things like C-SPAN and cable news generally, 1) leadership of committees and the parties consolidating more role-based power (especially in the House), 2) reduction in earmark-like things that enabled side deals to get bigger deals done, and 3) increased party polarization due to nationalization of party identities (measured through such things as DW-NOMINATE scores) were the key drivers,

davekasten 16 Dec 2025 18:21 UTC
10 points
2
in reply to: JohnofCharleston’s comment on: Toss a bitcoin to your Lightcone – LW + Lighthaven’s 2026 fundraiser
(Looks at half completed pitch deck titled “Starcology: A Lighthaven Managed Residence” and hits delete)

davekasten 15 Dec 2025 23:30 UTC
5 points
1
in reply to: AmandaFromBethlehem’s comment on: Toss a bitcoin to your Lightcone – LW + Lighthaven’s 2026 fundraiser
Obvious implication is obvious: Lighthaven should seriously consider ways to capture a share of value form “I have a flow of people moving to the Bay Area”. (Dumb version: find a realtor adjacent to this community, capture a share of referral fees. Other more sophisticated options are left as an exercise to the reader.)

davekasten 14 Dec 2025 19:17 UTC
26 points
8
on: Toss a bitcoin to your Lightcone – LW + Lighthaven’s 2026 fundraiser
I’m not making a one-time donation to Lighthaven, because I already have a recurring donation set up to Lighthaven. I encourage doing this as a general best practice for those donating ordinary USD or similarly boring funds, especially if your donation approach isn’t set up for a lump-sum approach.

While obviously less valuable in net present value terms than donating the full value upfront, having a predictable recurring fundraising flow from donors is quite useful for nonprofits.

davekasten 14 Dec 2025 18:55 UTC
22 points
4
on: Toss a bitcoin to your Lightcone – LW + Lighthaven’s 2026 fundraiser
Indeed, the traffic for AI 2027 in 2025 has been similar to the traffic for all of LessWrong in 2025, with about 5M unique users and 10M page-views, whereas LessWrong had 4.2M unique users and 22M page-views.
While my inner consultant loves this approach to impact sizing, I think another approach is to ask how much various political efforts would pay to get the Vice President (which on rough historical odds has ~50% chance of being the next President, in addition to his current powers) to read and publicly say that he’d read a document and had a positive opinion of it.

It is, uh, more like the think tank number than the paid ads number.

davekasten 30 Nov 2025 0:46 UTC
24 points
5
on: Unless its governance changes, Anthropic is untrustworthy
“in December 2024, Jack Clark tried to push Congressman Jay Obernolte (CA-23) for federal preemption of state AI laws” is a very strong claim, and one that I think is impossible for me to evaluate without context we don’t have.
I would encourage you to give context on what kinds of advocacy he was purportedly engaged in and what your sources allege to have believed the Congressman’s preferences on preemption were already at that time. I would not, for example, be especially surprised if the Congressman was already thinking hard about pushing for preemption at that time and Jack Clark was engaging him in a conversation where he had been made aware of (hypothetically) Congressman Obernolte’s plans. For example, I would be very dubious if you were claiming that Jack Clark came up with the idea and pitched it to Congress.

(I personally have no strong public opinion on preemption being good or bad in the abstract; the specific terms of what you’re preempting at the state level and what you’re doing at the federal level are most of the ballgame here.)

davekasten 20 Nov 2025 16:15 UTC
18 points
4
in reply to: Gretta Duleba’s comment on: Status Is The Game Of The Losers’ Bracket
For anyone reading this: if you’re ever in a situation where you wore a suit when others are less formal, Do Not Panic.

Remove tie (if wearing tie) and place in inside jacket pocket. Remove jacket and place out of view. Unbutton top button of collar, if buttoned. Roll up sleeves. Sit in a way that pants aren’t visible, if possible.

Boom, you’re now down to something much less formal in under 30 seconds.

davekasten 11 Nov 2025 18:36 UTC
4 points
0
on: One Shot Singalonging is an attitude, not a skill or a song-difficulty-level*
One thing to consider is a strategy used in Jewish singing contexts (which I see relatively rarely done in other contexts—but maybe it’s super common and I just don’t know the word to describe it!) Before singing the first verse or the chorus, you do a wordless verse where you’re just using nonsense syllables like “lai-dai-lai” or “yai-dah-dai-dai” that matches the song. This enables folks to pre-load the music before having to learn the words, and gives implicit social permission to, if you forget the words, to just do nonsense syllables that match the words. (A common problem if it’s a Hebrew text you’re unfamiliar with and you fall behind in reading it!)

(These are sometimes called niggunim, from the Hebrew for “tune” or “melody” for those wanting to google; for what should be fairly obvious reasons about a false cognate, I didn’t lead with that vocabulary term)

So for example, for The Circle, you’d start with something:
”Yah dai dai dai daiiii?
Dai dai, daaaai dah daaaai.
Yah dai dai dai daiiii?
Dai dai, daaaai dah daaaai.
Yah dai dai dai daiiii
Lah dah dai lai lai
Bah dah baiiih bah bah
Bah da bah baii bah
Bah da baiiii da baiii
Dai dai, daaaai dah daaaai.”
To presage the beat of verses like:
“So will we bring our families in,
Circle, grow and grow.
those whom Nature made our kin?
Circle, grow and grow.
Countless likenesses we find,
by our common blood bestowed.
What a debt of care is owed;
what a blesséd tie that binds!

Circle, circle, grow and grow.”

(The Circle is a little tricky in that the first verse starts slightly different, but trust me, this social technology extends to that use case as well, it’s not uncommon to have that in Jewish songs)

davekasten 20 Oct 2025 5:45 UTC
2 points
0
in reply to: Zach Stein-Perlman’s comment on: davekasten’s Shortform
To be clear, I think people should feel free to block freely and for any reason, including literally no reason at all. I’m open to ways of describing people’s block decisions in the future that better convey that, but I definitely didn’t think others reading this would assume “oh, Zach’s the bad guy here” as opposed to the reverse.

davekasten 20 Oct 2025 5:38 UTC
2 points
0
in reply to: habryka’s comment on: davekasten’s Shortform
By the “whitepaper,” are you referring to the RSP v2.2 that Zach linked to, or something else? If so, I don’t understand how a generic standard can “call out what kind of changes would need to be required” to their current environment if they’re also claiming they meet their current standard.
Also, just to cut a little more to brass tacks here, can you describe the specific threat model that you think they are insufficiently responding to? By that, I don’t mean just the threat actor (insiders within their compute provider) and their objective to get weights, but rather the specific class or classes of attacks that you expect to occur, and why you believe that existing technical security + compensating controls are insufficient given Anthropic’s existing standards.
For example, AIUI the weights aren’t just sitting naively decrypted at inference, they’re running inside a fairly locked down trusted execution environment, with keys provided only as-needed (probably with an ephemeral keying structure?) from a HSM, and those trusted execution environments are operating inside a physical security perimeter of a data center that already is designed to mitigate insider risk. Which parts of this are you worried are attackable? To what degree are organizational boundaries between Anthropic and its compute providers salient to increasing this risk? Why should we expect that the compute providers don’t already have sufficient compensatory controls here, given that, e.g., these compute providers also provide classified compute to the US government that is secured at the Top Secret / SCI level and presumably therefore have best-in-class anti-insider-threat capabilities?
I’m extremely willing to buy into a claim that they’re not doing enough, but I would actually need to have an argument here that’s more specific.

davekasten 19 Oct 2025 23:27 UTC
2 points
0
in reply to: habryka’s comment on: davekasten’s Shortform
I’m really not following your argument here. Of course in many instances compute providers don’t offer zero trust relationships with those running on their systems. This is just not news. There’s a reason why we have an entire universe of compensating technical and non-technical controls to mitigate risk in such circumstances.
You have done zero analysis to identify any reason to believe that those compensating controls are insufficient. You could incredibly easily get me to flip sides in this discussion if you offered any of that, but simply saying that someone’s running zero trust isn’t sufficient. As a hypothetical, if Anthropic is expending meaningful effort to be highly confident that they can ensure that Amazon’s own security processes are securing against insiders, they would have substantial risk reduction (as long as they can have high confidence said processes are continuing to be executed.
Separately, though it probably cuts against my argument above [1], I would politely disagree with the perhaps-unintended implication in your comment above that “implement zero trust” is a sufficient definition of defenses to defend against compute providers like Amazon, MSFT, etc. After all, Anthropic’s proper threat modeling of them should include things like, “Amazon, Microsoft, etc. employ former nation-state hackers who considered attacking zero trust networks to be part of the cost of doing business.”
[1] Scout mindset, etc.

davekasten 17 Oct 2025 3:52 UTC
2 points
−4
in reply to: MichaelDickens’s comment on: davekasten’s Shortform
Huh? Simply using someone else’s hosting doesn’t mean that Amazon has a threat-modeled ability to steal Claude’s model weights.

For example, it could be the case (not saying it is, this is just illustrative) that Amazon has given Anthropic sufficient surveillance capabilities inside their data centers that combined with other controls the risk is low.

davekasten 17 Oct 2025 3:50 UTC
2 points
0
in reply to: habryka’s comment on: davekasten’s Shortform
Where’s the “almost certainly” coming from? I feel like everyone responding to this is seeing something I’m not seeing.

davekasten 11 Oct 2025 18:28 UTC
11 points
0
on: davekasten’s Shortform
Zach Stein-Perlman’s recent quick take is confusing. It just seems like an assertion, followed by condemnation of Anthropic conditioned on us accepting his assertion blindly as true.

It is definitely the case that “insider threat from a compute provider” is a key part of Anthropic’s threat model! They routinely talk about it in formal and informal settings! So what precisely is his threat model here that he thinks they’re not defending adequately against?

(He has me blocked from commenting on his posts for some reason, which is absolutely his right, but insofar as he hasn’t blocked me from seeing his posts, I wanted to explicitly register in public my objection to this sort of low-quality argument.)

davekasten 30 Sep 2025 18:11 UTC
4 points
0
in reply to: Buck’s comment on: Buck’s Shortform
My opinion, FWIW, is that both treaty and international agreement (or “deal”, etc.) have upsides and downsides. And it’s hard to predict those considerations’ political salience or direction in the long term—e.g., just a few years ago, Republicans’ main complaint against the JCPOA (aka “the Iran Nuclear Deal”) was that it wasn’t an actual treaty, and should have been, which would be a very odd argument in 2025.

I think as long as MIRI says things like “or other international agreement or set of customary norms” on occasion it should be fine. It certainly doesn’t nails on the chalkboard me to hear “treaty” on a first glance, and in any long convo I model MIRI as saying something like “or look, we’d be open to other things that get this done too, we think a treaty is preferable but are open to something else that solves the same problem.”

davekasten 24 Sep 2025 22:39 UTC
5 points
0
in reply to: MalcolmMcLeod’s comment on: The title is reasonable
The big challenge here is getting national security officials to respond to your survey! Probably easier with former officials, but unclear how much that’s predictive of current officials’ beliefs.

davekasten 22 Sep 2025 4:24 UTC
8 points
7
on: The title is reasonable
I’m pretty sure that p(doom) is much more load-bearing for this community than policymakers generally. And frankly, I’m like this close to commissioning a poll of US national security officials where we straight up ask “at percent X of total human extinction would you support measures A, B, C, D, etc.”

I strongly, strongly, strongly suspect based on general DC pattern recognition that if the US government genuinely belived that the AI companies had a 25% chance of killing us all, FBI agents would rain out of the sky like a hot summer thunderstorm, sudden, brilliant, and devastating.