I’m writing a book about epistemology. It’s about The Problem of the Criterion, why it’s important, and what it has to tell us about how we approach knowing the truth.
I’ve also written a lot about AI safety. Some of the more interesting stuff can be found at the site of my currently-dormant AI safety org, PAISRI.
I think there’s a hidden variable in this framework: the effective branching factor of the task. For well-specified tasks (prove this theorem, implement this API to spec, etc.), AI cost probably does scale roughly linearly with task size, matching your model. But most real-world engineering tasks aren’t like that. The bottleneck isn’t execution speed, it’s the iterative discovery of what the task actually is.
A senior engineer doing a 4-hour task isn’t typing for 4 hours. They’re making dozens of micro-decisions informed by tacit knowledge about the codebase, the product, the users, the org. They’re constantly pruning the search space. An AI working autonomously on the same task produces tokens faster but explores a much larger space of wrong approaches, because it lacks that contextual judgment. So the cost might scale superlinearly with task complexity not because of compute limits but because of specification limits.
This suggests the linear regime on your Pareto frontier might be shorter for real-world task distributions than for benchmark-style tasks, and the “fraction of human cost” metric could look very different depending on how well-specified the task is. The tasks where AI efficiency looks great (tight feedback loops, clear success criteria) are not the same tasks that fill most of an engineer’s day.
(Though note that obviously this could change if AI starts to have more context and better judgement, which would basically mean training a better model.)
Yes.
For me the actual experience of resolving the Great Matter felt like dying. It’s not a coincidence that some folks call it Dying the Great Death. That part was not too dissimilar from the ego death that can be experienced with various drugs, though with the difference that it feels less like being temporarily blown apart and more like a permanent dying of something.
But after that I spent several months with easy access to a feeling of bliss. The bliss eventually gave way to tranquility and later equanimity. As I often describe it, every moment of every day is a 9⁄10 (and only not a 10⁄10 because I know I give up some feeling good to be actively living in the world), even when I’m quite sad or upset or anxious or anything else. Relatively speaking, though, this means that most of the time things are “just fine” or “okay” because there’s not the constant fluctuation between suffering and release.
I do want to caveat this answer, though, by saying that the process to get here involved the worst suffering of my life. Long weeks of hopeless despair, years of confusion and dissatisfaction. I’m only happy now because I went through all that, and I only went through it all because I had no choice—I couldn’t find a way to be happy without going through it.
My general philosophy on cooking is “if it can’t be cooked it in the microwave, then I ain’t cooking it”, so this post is right up my alley!
I’m guessing you never tried to cook eggs in a microwave before. The exploding egg problem is relatively well known. The typical solution is to puncture the egg to release the steam that builds up between the white and the yolk, since that’s what causes the explosion. They even sell special dishes for making poached eggs in the microwave.
Also, pasta is incredibly easy to cook in the microwave. Again, they sell a special dish for it, which is basically just a long plastic rectangle so you can cook long noodles. Shorter noodles are also allowed. The main challenge is keeping the thing from boiling over, which can be achieved with a combination of salted water, oil, and not trying to cook too much at once. Pasta will cook in water that’s not at a roiling boil, though it takes longer, but the end result is about the same.
Rice, potatoes, carrots, and lots of other things are also quite easy to cook in the microwave. Even fish is pretty easy to cook in one, although I don’t recommend it because it makes a lot of fish smell (unless you like that).
Thanks for the feedback. I tend to be an abstractor, as I think of it: gathering data and building abstractions that structure it. This means my output tends to be to tell you about cool abstractions I figured out.
I’m less excited about performing the opposite task, which is presenting raw data for others to build abstractions on. I mostly only do it when I feel like, rhetorically, I’m going to lose the reader if I don’t. I freely admit this is a weakness on my part and I’m trying to get better about providing data in posts, but I admit I’m not excited about it.
If you want something more like Chapman’s most recent post, Kegan’s In Over Our Heads is really good: it tells a stylized story of a person going through the struggle of the stage 3 to 4 transition and is based on an amalgamation of real stories (aggregated and anonymized for privacy). Or, alternatively, I’ve gotten a lot of milage out of both slice-of-life type stories that follow realistic characters and reading memoirs and other true stories from people. I have to do the work of applying whatever theory I’m trying to work out to the story, but learning more about the experiences of many people is the raw data necessary for making sense of a great many theories (and for formulating your own!).
My perhaps extremely obvious question is, why is this necessary?
I’m not opposed to finding new ways to explain old ideas to Western audiences, but why do we need to dress it up in the language of complexity theory? All Buddhist theory ultimately exists for the purpose of helping point people to the way to awaken for themselves. Maybe there’s some people who need it explained in terms of complexity theory to make sense of it, but I suspect there’s other, more familiar and more accessible metaphors that would help a larger number of people. Also, as ever, there’s some risk in misunderstanding when translating to another ontology, and I’m not sure if translating to complexity theory results in conveying the same connotations, which might result in subtle confusions that are hard to tease out.
What I’d like to point you to is my book, Fundamental Uncertainty. Alas, it’s still in revisions and not published yet, though I did put the first draft up on LessWrong to get feedback. It ultimately makes this case, but does so by gently building up to it. If you want to skip ahead, most of the “rationalist are screwing up; here’s how to do better” content is in Chapters 8 and 9 (though because I target a wider audience, rationalists are not explicitly named).
An older post that I no longer fully endorse that’s in the vein you’re looking for is: https://www.lesswrong.com/posts/5LP6Jc8ztwcyb296X/outline-of-metarationality-or-much-less-than-you-wanted-to
But maybe the best take on this is not mine, but David Chapman’s Meaningness. I’d recommend starting with this essay.
Yes. Specifically I was building agents to play games as part of a beta with SoftMax.
Sadly, it’s too abstract a warning shot.
I think a real warning shot that’s actually registered as such by the public and politicians would have to be something that involves a lot of people dying or a lot of economic damage. Otherwise, I have a hard time seeing a critical mass of people finding motivation to act.
Well, as I say in my example above, literally build a bot that plays a game.
Most of the loops end up much shorter, though, like “upgrade this package dependency, keep fixing bugs in the build until the build passes”, but sometimes these changes are kinda weird, so I try to get Claude to do what a human would do, which is keep trying things it thinks might work to get the build to pass.
Or, one I haven’t done but might: keep adding tests until we hit X% coverage (and give some examples of what constitutes a good test). This one I expect to work better than you might think, since Opus is getting reasonably good at not specification gaming and trying to actually do what I mean, which Sonnet frequently still goes for.
You need a clear measure. For example, let’s say you want to build a scripted bot that can play a novel game for which there is not an off the self solution. You could try to train a neural net, but Claude can write code, so you fill in Y with “writing a bot that plays game Z”.
This sort of strategy is obviously heavily dependent of the availability of a good evaluation method and a clear scoring mechanism. As such, it doesn’t work for most problems, since most problems don’t have such large search spaces.
Yes, Opus 4.5.
Perhaps a point of terminology, I’d say vibestemics is itself about the fact that your epistemics, whatever they are, are grounded in vibes (via care). However, this is tangled up with the fact that to believe that this core vibestemic claim is true is to automatically imply that there is no one right epistemic process, but rather epistemic processes that are instrumentally useful depending on what you care about doing (hence the contingency on care).
The specific vibe of the post-rationality is, as I would frame it, to value completeness over consistency, whereas traditional rationality makes the opposite choice (and pre-rationality doesn’t even try to value either, except in that it will try to hallucinate its way to both if pressed).
Giving Claude looping instructions can be quite useful. But I never go full Ralph Wiggum!
For example, here’s a paraphrase of a loop I had Claude run recently with --dangerously-skip-permissions:
keep iterating on this code in a loop. think of yourself as a scientist. come up with hypotheses, run experiments, see what works, and iterate. keep going until we get at least a score of X at task Y. i know it’s possible, you can do this, i believe in you, let’s go!
5 hours of clock time later it had done very well. :-)
I don’t have great faith in the epistemics of postrats as they exist today.
Yeah, you and me both.
I’ve said this elsewhere before, but in hindsight it was a mistake for us to promote terms like “postrationality” and “metarationality” to the point of fixation. They’re exactly the type of words that invite pre/post confusion and allows pre-rats to masquerade as post-rats if there’s insufficient gatekeeping (and there usually is).
And yet, there’s something in the desire of folks like myself to point to a place that says “hey, I think rationalists are doing a lot of things right, but are screwing up in fundamental ways that are contrary to the vibe of rationality, and it’s useful to give that thing a name so we can easily point to it”.
In my ideal world, people would be trained in rationality-as-it-exists-today first, and then be trained in the limits of those methods so they know how to transcend them safely when they break down. Then post-rat would really mean something: one who fully trained as a rationalist, and then used that as the bedrock on which to learn how to handle the situations the methods of rationality are not good at dealing with.
Some people will argue that’s just rationality, and sure maybe it is some ideal version of rationality as proposed in The Sequences, but as I see it, actual rationalists screw up in predictable ways, those ways are related to the rationalist vibe, and thus the internal experience must be to transcend that vibe, whatever we want to label it.
That’s my hope.
I’m saying he’s right to say that the friction is unresolvable. I’m sure it does feel mysterious, but it’s actually very straightforward in a way that’s difficult to explain unless you can already see it. I wish it wasn’t like this. But it’s true that you both keep developing and that you’ve attained something (and the thing you attain is, in part, realizing there was nothing to attain in the first place!). Steve Byrnes has done a decent job of trying to explain it as well as anyone has.
It also sounds like you’re saying that, even after reaching enlightenment, you’ll still have mental habits that become maladaptive over time. That’s interesting, it wasn’t my impression of what enlightenment was like.
“Enlightenment” is a rather slippery word, and I’ve of the opinion that some people are intentionally slippery about it because it benefits them. So some people use the word to mean both that you’ve had a persistent realization of non-duality (that’s what PNSE is about) and that you are free of all preconditioned reactions (you’re liberated). But almost no one is fully liberated (and to stay that way you seemingly have to live a very constrained life that protects you from interacting with the world), and lots of people are in PNSE, so it’s probably more useful for “enlightenment” to be about PNSE than it is for it to be about a state of PNSE plus zero reactivity.
(Policing who can claim to be “enlightened” is a centuries old issue in Buddhism, and many lineages have developed social systems to deal with it.)
Firefighters ? Medical emergencies ? Police ? Ambulance ? You should have the gesture of calling them with a phone carved in your brain (see here for a refresher).
During a real emergency, dialing 911 can be surprisingly difficult. You’re emotionally frazzled, possible panicking, and just hitting the right sequence of buttons on the phone can become hard (this was easier before we had to do things like unlock our phones or remember the sequence to do to bring up a screen to dial 911 without unlocking). I don’t know of a great solution here, since to train you actually have to dial 911, but it’s something to be aware of.
Yes, very much so.
I can remember being 4 years old, laying in bed, scared I would die in the night and wake up a skeleton. Only, then I would realize what it would mean to be dead, and I not only couldn’t really conceive it except abstractly, but felt deep fear even trying to conceive it. But, being the kind of person I am I guess, I made myself look at it for as long as I could. It didn’t really work, but I’d try.
You could say I had to try to stare into the face of my own death for 37 years until I was finally able to see it.