Communications consultant for AI safety and policy.
drgunn
Karma: 46
On the topic of capabilities, agentic research assistants have come a long way in 2025. Elicit & Edison Scientific are ones to watch, but they still struggle to adequately cite the sources for the claims in the reports they generate our conclusions they come to. Contrast this with Deep Research models, which include nonsense from press releases and blog posts, even when explicitly asked to exclude these, though they have improved significantly too. Progress, but we’re still a long way from what a average PhD could put together. They sure do it more quickly, but moving more quickly in the wrong direction isn’t that helpful in research! One encouraging direction I see some tools (notably Moara.io) moving is towards automating the drudgery of systematic data extraction, freeing up experts to do the high-context analysis required for synthesizing the literature.
If you’re not an expert, you’ll no doubt be impressed, but be careful. Gell-Mann amnesia is still as much a thing with LLM-assisted research as it ever was with science journalism.
It just takes time. Your diaphragm will eventually stop being so annoying.
Not in my experience, which is substantial, but perhaps specific to me.
Good advice, but note that it doesn’t apply to all running, but specifically to street or track running. Trail is different in a few notable ways.
Major differences:
You want more of a forefoot strike. Your stride will be more like the motion of pedaling a bike, where your front foot is actually coming back in when it hits the ground. This is particularly important going downhill, as it reduces load on the knee & tendons. Likewise, if you leave a little room for your back foot to go back at the end of the stride, you can save yourself if you get snagged on a root or something.
Get a low heel-toe drop trail shoe to facilitate this stride, improve traction, and facilitate better ground feel.
Let your arms hang at your sides. It feels weird, but on longer runs (2 hrs+), your biceps will not like the bounce that comes with the bend.
Epistemic status: generally aligned with pro recommendations, injury free for 15 years, starting once I adopted these practices.
I’m really glad to see taste enter the discourse! Taste is a meta-rational skill and meta-rationality—the art of relevant realization, how to not get stuck optimizing up to a local maximum that’s not actually where you want to be—has very much been lacking in public conversation about how to get more of the good AI can bring while also doing common-sense things to manage risk.
Why have I been concerned about this? Optimizing up to a local maximum which isn’t actually where you want to be is EXACTLY the problem that so many content platforms (Facebook, Twitter, Quora) have gotten themselves into and my impression has been that the people building AI products are drawn from the same population. So seeing taste get discussed is very welcome and, if this continues, will definitely prompt a positive update to my expectation of how things are going to turn out.
A useful direction for further research would be to analyze fiction and non-fiction as separate corpora. Longer sentences may reflect a tendency towards hypotaxis, and they may also be a deliberate stylistic choice for fiction writers, perhaps setting a more languid and relaxed tone not appropriate for much non-fiction.
Fanelli is a good, if dated reference for this. Another important point is that there are levels of misconduct in research, ranging from bad authorship practices to outright fabrication of results, with the less severe practices being relatively more common: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4269469/
Aside from all that, there’s irreproducibility, which doesn’t arise from any kind of deliberate misconduct, but still pollutes the epistemic commons: https://www.cos.io/rpcb
As someone with experience in BSL-3 labs, BSL feels like a good metaphor to me. The big issue with the RSP proposal is that it’s still just a set of voluntary commitments that could undermine progress on real risk management by giving policymakers a way to make it look like they’ve done something without really doing anything. It would be much better with input from risk management professionals.
I’m confused.The protest is now listed at 4. Have you coordinated with them?
I like it, but it feels like you could have worked snakes in there somehow: https://www.vectorsofmind.com/p/the-snake-cult-of-consciousness
X-risk discussions aren’t immune from the “grab the mic” dynamics that affect every other cause advocacy community.
There will continue to be tactics such as “X distracts from Y” and “if you really cared about X you would …” unless and until people who care about the cause for the cause’s sake can identify and exclude those who care about the cause for the sake of the cultural and social capital that can be extracted. Inclusivity has such a positive affect halo around it that it’s hard to do this, but it’s really the only way.
Longer-form of the argument: https://meaningness.com/geeks-mops-sociopaths
When I hear someone saying,
“Because it is asking to me to believe it completely if I believe it at all, I feel more comfortable choosing to consider it “false” on my terms, which is merely that it gave me no other choice because it defined itself to be false when it is only believed weakly.”
what I think is “of course there are strong and weak beliefs!” but true and false is only defined relative to who is asking and why (in some cases), so you need to consider the context in which you’re applying LoEM.
In other words, LoEM applies to “Does 2+2=4?” but it does not apply to “Is there water in the fridge?”, unless the context is specified more carefully.
It’s obviously an error to only have 100% or 0% as truth values for all propositions, and it’s perhaps less obviously an error to have the same probabilities that a proposition is true across all possible contexts in which that proposition might be evaluated.
More here: https://metarationality.com/refrigerator
You seem to be talking about “combinatorial explosion”. It’s a classic problem in AI, and I like John Vervaeke’s approach to explaining how humans solve the problem for themselves. See: http://sites.utoronto.ca/jvcourses/jolc.pdf
No one has solved it for AI yet.
It’s a very interesting model of the world that tastes 18 different brands of store-bought sauce and doesn’t compare it to just making your own. Add “seriouseats” to any recipe-related query and you’ll get recipes that both work and taste like they’re supposed to. They may eventually fire their writers and replace them with AI trained on blogspam recipes, so exploit this knowledge while you can. Surprisingly few people know about it, given how much utility basic culinary knowledge can add to your life.
What if AI safety and governance people published their papers on Arxiv in addition to NBER or wherever? I know it’s not the kind of stuff that Arxiv accepts, but if I was looking for a near-term policy win, that might be one.
It strikes me that 80000 hours puts you just about when the prediction markets are predicting AGI to be available, i.e., a bit late. I wonder if EA folks still think government roles are the best way to go?
I think this is a very clear explanation of how to move from initial success to scalable impact, which is something that every new organization struggles with. Thanks for writing this up and sharing it!
I also liked the part about focusing on the real goal. There’s a growing coalition of organizations in the US that are focusing a lot on data center opposition as a means to an end, and I feel like that’s going to lead to some slowing down of the rate of progress, but it also feels a little like what the environmental movement did.
Can’t wait to see what ControlAI gets up to next!