Daniel Kokotajlo

Was a philosophy PhD student, left to work at AI Impacts, then Center on Long-Term Risk, then OpenAI. Quit OpenAI due to losing confidence that it would behave responsibly around the time of AGI. Now executive director of the AI Futures Project. I subscribe to Crocker’s Rules and am especially interested to hear unsolicited constructive criticism. http://​​sl4.org/​​crocker.html

Some of my favorite memes:

(by Rob Wiblin)

(xkcd)

My EA Journey, depicted on the whiteboard at CLR:

(h/​t Scott Alexander)

• Daniel Kokotajlo 2 May 2026 4:55 UTC

  13 points

  7

  in reply to: StanislavKrym’s comment on: How Go Play­ers Disem­power Them­selves to AI

  Oh I don’t think it would.
  
  Yeah a thing I was alluding to is that maybe in other domains, e.g. coding, there’ll be a period where lots of people use AI for coding and they tell themselves that it’s still them doing the coding when really they are kinda useless middle managers between their boss and Claude, and they tell themselves they are learning new languages etc. when really their coding skills are starting to atrophy.

• Daniel Kokotajlo 1 May 2026 23:46 UTC

  23 points

  11

  on: How Go Play­ers Disem­power Them­selves to AI

  Thank you for this post, I found it helpful and interesting. I agree that this is a data point of evidence about how AI automation in other domains might go down.

• Daniel Kokotajlo 1 May 2026 21:37 UTC

  4 points

  0

  in reply to: Linch’s comment on: Linch’s Shortform

  What prompt do you use?

• Daniel Kokotajlo 1 May 2026 16:54 UTC

  2 points

  0

  in reply to: Daniel Kokotajlo’s comment on: Thomas Larsen’s Shortform

  Update: Semianalysis claims:

  

• Daniel Kokotajlo 1 May 2026 16:41 UTC

  17 points

  1

  in reply to: Joseph Bloom’s comment on: Joseph Bloom’s Shortform

  IMO, UK AISI is kicking ass and one of the best places to have positive impact on making AGI go well. I too encourage people to apply!

• Daniel Kokotajlo 1 May 2026 4:07 UTC

  16 points

  16

  in reply to: anaguma’s comment on: anaguma’s Shortform

  Just because they haven’t drawn that lesson yet doesn’t mean this data point won’t be remembered and contribute to their eventual drawing of that lesson.

• Daniel Kokotajlo 30 Apr 2026 16:28 UTC

  9 points

  1

  in reply to: Aaron Staley’s comment on: Thomas Larsen’s Shortform

  I think those examples actually reinforce Josh’s point. NVIDIA growth is also from AI. Zoom and Moderna grew because of COVID creating a ginormous demand shock, yet even then, their growth at its peak was slower than Anthropic’s growth despite them starting from a smaller revenue level and therefore it being inherently easier for them to grow. So… unless you can dig up better examples, it seems like Anthropic’s Q1 2026 growth is literally the most impressive growth of any company in history? And this is despite the fact that there’s no COVID-equivalent for AI; there’s no unusual circumstance that created a huge temporary demand shock, instead, it’s just that they made their products better.

• Daniel Kokotajlo 30 Apr 2026 5:10 UTC

  3 points

  0

  in reply to: Steffee’s comment on: Fore­cast­ing is Way Over­rated, and We Should Stop Fund­ing It

  We probably don’t have a foolproof way to distinguish but there would be various other bits of evidence we could look for, e.g. how capable are the AIs actually getting, what’s the gossip in SF, how long and sustained has the spike been, etc.

• Daniel Kokotajlo 30 Apr 2026 2:33 UTC

  2 points

  1

  in reply to: rahulxyz’s comment on: Thomas Larsen’s Shortform

  I agree actually, that maybe AIs not too different from today’s could get to $10T after proliferating into the economy for another decade.
  
  So perhaps Thomas’ argument should be revised to more specifically be about the next two years or so. If Anthropic or OAI make it to $10T by 2029, then that seems like something that couldn’t be achieved with just slightly better versions of current AIs. There just isn’t enough time to build all the products on top of it, transform all the industries, outcompete the dinosaurs, etc. Whereas if they actually do have a drop-in replacement for human professionals at everything, then yes they’d make it to $10T.

• Daniel Kokotajlo 29 Apr 2026 22:59 UTC

  LW: 10 AF: 5

  4

  AF

  in reply to: Charlie Griffin’s comment on: Char­lie Griffin’s Shortform

  I was already impressed by UK AISI but I keep reading things that make my opinion of y’all rise still further. Keep on doing whatever it is that you are doing!

• Daniel Kokotajlo 28 Apr 2026 21:32 UTC

  4 points

  1

  in reply to: rahulxyz’s comment on: Thomas Larsen’s Shortform

  Speaking for myself:

  First of all, the implication “AGI --> Loads of revenue” does seem to hold. If one of these companies did get to AGI, they’d pretty quickly get to 1T, then 10T, then 100T ARR.
  
  What about the implication “Loads of revenue --> AGI?” That’s trickier. But the basic intuition is that in order for a company like Anthropic to be making 10T ARR, they must be deploying Claude pretty much across the whole economy. Claude must be embedded in basically everything, and providing a lot of value too, otherwise people wouldn’t be paying 10% of world GDP for it. And it seems like a Claude capable enough to provide that much value to that many different diverse industries, would probably be AGI. If there was still some major skill/​ability that it lacked, some major way in which humans were superior, then probably that deficiency would prevent it from making $10T ARR, by limiting it to certain industries or roles that don’t require that skill/​ability.

• Daniel Kokotajlo 28 Apr 2026 20:28 UTC

  8 points

  0

  in reply to: Hoagy’s comment on: Thomas Larsen’s Shortform

  Well, I certainly was surprised! So, guilty as charged I guess? I still stand by the comment. What parts of it do you disagree with?

• Daniel Kokotajlo 28 Apr 2026 20:09 UTC

  7 points

  0

  in reply to: niplav’s comment on: ni­plav’s Shortform

  Related prior writing on this subject by me: https://​​forum.effectivealtruism.org/​​posts/​​Z6Ssc79vH496bLqL9/​​tiny-probabilities-of-vast-utilities-a-problem-for-long

• Daniel Kokotajlo 28 Apr 2026 18:45 UTC

  28 points

  −1

  in reply to: Thomas Larsen’s comment on: Thomas Larsen’s Shortform

  Note that in order for Anthropic revenue to 10x this year, they’ll already have to increase $/​FLOP (i.e. revenue per unit of compute. Profit margins basically.) To increase it another 10x the following year, they’ll probably need to triple $/​FLOP, because their compute will only roughly triple next year. Ditto for 2028. All this is a reason to doubt premise 1 basically; in the past they’ve been able to grow revenue in large part by just allocating more of their compute to serving customers, but now they’ll have to charge customers more per FLOP.

• Daniel Kokotajlo 28 Apr 2026 16:01 UTC

  2 points

  0

  in reply to: yams’s comment on: Eric Ney­man’s Shortform

  I’m more optimistic than you I guess.
  
  Remember Liberation Day? Trump’s Tariffs? The reaction of the stock market was crucial for determining how all that played out; things would have been much worse if not for the clear unambiguous signals being sent by the market about the expected future effect of various announcements. I count this as a win for prediction markets, because stock markets are basically a form of prediction market about the value of companies. Seems plausible to me that we can use prediction markets to bring the magic of stock markets to other domains, and that if we did that, it would be really good.

• Daniel Kokotajlo 27 Apr 2026 21:00 UTC

  6 points

  0

  in reply to: Steffee’s comment on: Fore­cast­ing is Way Over­rated, and We Should Stop Fund­ing It

  Yes, but there are also some examples of uncovering little-known info, e.g. insider trading giving advance warning of stuff.
  
  Consider: Suppose that someone tries to assassinate POTUS by running up with a suicide vest on. Suppose that it appears they were a random ideologue acting alone, but then analysis of the markets reveals a suspicious spike in bets on POTUS death starting a few hours before the attack. This suggests that this person didn’t act alone; it suggests that at least one other person knew about it beforehand. Indeed it suggests that many people knew, because if it was just a few people then the odds that someone would leak are low. This is some interesting info that wouldn’t otherwise have been uncovered! (And if the Secret Service is monitoring the markets, they might actually be able to protect POTUS from such attacks more effectively!)

• Daniel Kokotajlo 27 Apr 2026 18:21 UTC

  6 points

  2

  in reply to: Eric Neyman’s comment on: Eric Ney­man’s Shortform

  Brainstorming more examples:

  --China and US are in a tense standoff, negotiating a possible AI Deal to end the AI race. Tech company propagandists are bleating that if POTUS accepts the latest proposal, China will win and we’ll all be slaves forever. Prediction markets 40% credence to “deal accepted” claim and 4% probability to the “China wins conditional on deal acceptance?” claim, because traders correctly realize that actually things will be pretty normal if the Deal is accepted. They also assign 6% probability to “China wins conditional on Deal rejection” claim because they assign 50% probability to “WW3 if no Deal by EOY” and they think China has a 24% chance of winning WW3 and that if this Deal is rejected then there’s a 50% chance that no Deal will be made by EOY.
  
  --There’s a round of mass layoffs in some sectors, and lots of people freak out about AI unemployment. Then there’s a backlash to the freakout, with lots of economists and tech shill types saying that actually things will be fine just like previous automation waves. However, the prediction markets aren’t buying it; they say that a year from now unemployment will be above 50% with 75% probability. “If you’re so smart” people say to the economists “why don’t you 4x your money in a year by betting on the markets?”
  
  (I could get back to work but I suspect I could do this all day)
  
  
  
  

• Daniel Kokotajlo 24 Apr 2026 17:44 UTC

  11 points

  0

  in reply to: ChristianKl’s comment on: Daniel Koko­ta­jlo’s Shortform

  I agree that their priority isn’t making money. But prices are set by supply and demand, not by how much companies want to make money. The demand is going to go up a lot as AI becomes more actually useful in the real world, and the supply is going to go up only a little bit as the companies triple their total compute.

• Daniel Kokotajlo 24 Apr 2026 5:50 UTC

  51 points

  18

  on: Daniel Koko­ta­jlo’s Shortform

  Seems like AI is going to get more expensive over the next year. GPU prices are spiking a bit in the last two months, Claude users are complaining about usage limits more than usual, Mythos is supposed to be more expensive, Anthropic revenue has been growing at an insane pace (10x a year for more than two years) and seems like even as it slows down it’ll still grow faster than they can scale compute. All these signs point in the same direction: More expensive frontier AIs. The free-tier chatbots and similar will still be cheap, but if you want the best AIs in December you’ll have to pay a lot more than people are used to paying today. (Even on a per-token basis; since you’ll probably also be asking them to do longer, more ambitious tasks, the price increase will seem even bigger)
  
  Or so it seems at least.

• Daniel Kokotajlo 24 Apr 2026 5:26 UTC

  LW: 7 AF: 3

  0

  AF

  in reply to: Sam Marks’s comment on: Cur­rent AIs seem pretty mis­al­igned to me

  If you would have predicted 15% for Agent-2, what would you have predicted for Agent-1 and Agent-0 levels? Presumably less than 15%?