contact: jurkovich.nikola@gmail.com
Nikola Jurkovic
Karma: 2,912
Results of a small ZBiotics RCT
Now seems an especially low-cost time for AGI company employees to make statements in support of pauses.
Over the last week, there has been an unprecedented level of support for a pause/slowdown from AGI companies. A few examples:
An autonomous humanoid robot beat the human world record time for a half-marathon this year in Beijing. A robot called Lightning (by Huawei subsidiary Honor) finished in 50 minutes and 26 seconds, compared to the human world record of 57 minutes and 20 seconds.
Last year, the best time for an autonomous human robot was 3 hours and 37 minutes. (wiki)
To see it themselves, I think it’s a good movie that could be motivating (and a fun time to watch) to a lot of young people. I think it’s especially good for people who are kind of interested in AI safety but haven’t fully decided whether to work on it or not (many members of AI safety student groups fit this description).
AI safety student groups should probably make it easy for their members to see The AI Doc in the cinema. Maybe announce that you’ll all go to the same screening on a wekeend, offer to cover tickets, have a thing right after where you talk about it and maybe get free food or something
I would be really interested to see the results of other companies’ models on this!
If you have a Costco membership you can buy $100 dollars of Uber gift cards online for $80. This provides a 20% discount on all of Uber.
Sadly you can only buy $100 every 2 weeks, meaning your savings per year are limited to 20$ * (52/2) = $520. A Costco membership costs $65 a year. It’s unclear how long this will stay an option.
I think that space-based power grabs are unlikely as long as powers care about, and are equally-matched on, Earth.
This is the rough story that I think is unlikely to happen:
Two superpowers have roughly equal power on the Earth during the singularity, and remain roughly equal in power after both creating ASIs that are at least intent-aligned with them. They maintain mutually assured destruction on Earth. Superpower A is much more focused on building space infrastructure than Superpower B. Within a decade, Superpower A’s space infrastructure means that Superpower A has a decisive advantage superpower B.
This story to me seems unlikely because in this scenario, Superpower A probably still has most of its human population on Earth (relocating millions of people to space would probably be very slow). Therefore, as long as mutually assured destruction is maintained on Earth, Superpower B will retain a lot of its bargaining power despite having a disadvantage in space infrastructure.
Thank you for writing this, I find it very relatable. I’d heart react the post if that feature existed, so I’ll heart react my comment instead.
This benchmark includes a Slay the Spire environment! When it was written, Gemini 2.5 did the best, getting roughly halfway through a non-Ascension run.
This very roughly implies that the median of “50% time horizon as predicted by METR staff” by EOY 2026 is a bit higher than 20 hours.
I very roughly polled METR staff (using Fatebook) what the 50% time horizon will be by EOY 2026, conditional on METR reporting something analogous to today’s time horizon metric.
I got the following results: 29% average probability that it will surpass 32 hours. 68% average probability that it will surpass 16 hours.
The first question got 10 respondents and the second question got 12. Around half of the respondents were technical researchers. I expect the sample to be close to representative, but maybe a bit more short-timelines than the rest of METR staff.
The average probability that the question doesn’t resolve AMBIGUOUS is somewhere around 60%.
I think my median is now 4 years, due to 2025 progress being underwhelming. I plan to write a follow up post sometime soon.
I don’t endorse the timelines in this post anymore (my median is now around EOY 2029 instead of EOY 2027) but I think the recommendations stand up.
In person, especially in 2024, many people would mention my post to me, and I think it helped people think about their career plans. I still endorse the robustly good actions.
How did my 2025 predictions hold up? Pretty well! I plan to write up a full post reviewing my predictions, but they seem pretty calibrated. I think I overestimated public attention, frontiermath, and I slightly overestimated SWE-Bench verified and OSWorld. All of the preparedness categories were hit I think.
College life with short AGI timelines
Diplomacy during AI takeoff
Are AI time horizons inherently superexponential?
How likely is dangerous AI in the short term?
I don’t see why either of those things stop you from having a family.
I think we might be using different operationalizations of “having a family” here. I was imagining it to mean something that at least includes “raise kids from the age of ~0 to 18″. If x-risk were to materialize within the next ~19 years, I would be literally stopped from “having a family” by all of us getting killed.
But under a definition of “have a family” which is means “raise a child from the age of ~0 to 1″, then yeah, I think P(doom) is <20% in the next 2 years and I’m probably not literally getting stopped.
Also to be clear, my P(ASI within our lifetimes) is like 85%, and my P(doom) is like 2⁄3.
We appear to have entered a de facto government-enforced pause on making AIs more cyber-capable (it’s not quite a hard pause internally, but making it illegal for non-US people to use mythos-equivalent models is probably extremely inconvenient). It’s based on something like the condition that the AIs need to be unjailbreakable which seems really hard. I’m surprised we entered any sort of AI pause in 2026.