I am a PhD student in computer science at the University of Waterloo, supervised by Professor Ming Li and advised by Professor Marcus Hutter.
My current research is related to applications of algorithmic probability to sequential decision theory (universal artificial intelligence). Recently I have been trying to start a dialogue between the computational cognitive science and UAI communities. Sometimes I build robots, professionally or otherwise. Another hobby (and a personal favorite of my posts here) is the Sherlockian abduction master list, which is a crowdsourced project seeking to make “Sherlock Holmes” style inference feasible by compiling observational cues. Give it a read and see if you can contribute!
See my personal website colewyeth.com for an overview of my interests and work.
I do ~two types of writing, academic publications and (lesswrong) posts. With the former I try to be careful enough that I can stand by ~all (strong/central) claims in 10 years, usually by presenting a combination of theorems with rigorous proofs and only more conservative intuitive speculation. With the later, I try to learn enough by writing that I have changed my mind by the time I’m finished—and though I usually include an “epistemic status” to suggest my (final) degree of confidence before posting, the ensuing discussion often changes my mind again. As of mid-2025, I think that the chances of AGI in the next few years are high enough (though still <50%) that it’s best to focus on disseminating safety relevant research as rapidly as possible, so I’m focusing less on long-term goals like academic success and the associated incentives. That means most of my work will appear online in an unpolished form long before it is published.
I’ve sporadically worked on ML including in industry and it has done much less than you’d expect to inform my views on the risk of extinction from ASI. (I can’t rule out that I simply didn’t go deep enough to get the benefits e.g. I’ve never done ML engineering as a full time job.)
Top ML engineers do gain important intuitions that the authors might be missing, but those intuitions are IMO mainly about how train better models (e.g. taking better advantage of GPU parallelization) and not how to control a superintelligence. In many cases engineers just haven’t thought about the risks or have highly naive takes (though this seems to be less true at the top). I think part of the reason is that the process is now so “industrialized,” there’s no longer much theory behind pushing the frontier—it’s about squeezing as much performance as possible out of known techniques. The picture I’m trying to gesture at here is similar to if you were growing a superbrain in a vat, the engineers would be mainly vat-engineers, specializing in building large vats and choosing the right chemicals, and at some point maybe just one of those two. The situation is not that extreme, but it’s getting closer. (And this impression is based on conversations with career ML engineers, who I interact with very frequently)
I’d certainly trust the views of someone who has thought deeply about the issue AND worked on the systems the most (an example is Rohin Shah). Roughly speaking, the former teaches you security mindset (not to try naive ideas that will obviously break) and the later teaches you what can(not) be done in practice, which shoots down some “theoretically” appealing plans.
However, it’s hard to blame Yudkowsky and Soares for choosing not to work at the frontier labs which they believe are currently pushing us closer to extinction, and the competence which they would IDEALLY have is kind of hard to get in another way (which means it’s hard to find better qualified representatives of this particular worldview—this should in the Bayesian sense somewhat “screen off” their missing credentials when assessing their credibility).