LessWrong Team
I have signed no contracts or agreements whose existence I cannot mention.
Ruby
Karma: 15,925
Well, steam engines have even less coherent world models.
I believe in their power from seeing just how much value they give me and how transformative they are for me. I’m a super early adopter, but if I extrapolate the rest of the world making as much use of the tech as I am, and doing all the things I could see doing, it’s still so much.
Your Claude transcript covers the relevant response:
Meanwhile, a person grabbing a wheel at the studs (which are maybe 2–3 inches from center on a typical bolt pattern) is actually at a disadvantage compared to grabbing the rim. At the studs, your lever arm is very short. If you’re gripping at roughly 2.5 inches from center and pulling hard with maybe 50–80 lbs of force, that’s only about 10–17 ft-lbs of torque. That’s dramatically less than the hill torque.
So the writer may actually be correct for the specific scenario they described — trying to turn the wheel at the studs rather than at the rim. That’s a crucial detail.
I do update that the amount of torque the car is experiencing under gravity is more like 150-200ft-lb and therefore closer to what a human can produce with a good lever arm. Though my Claude’s assertion was “a lot less than someone deliberately trying to wrench a wheel around”, which is not true even with more leverage – they are perhaps comparable then.
Regarding case 2, Claude knew we were just running on my Macbook where the marginal cost of running is negligible, and from my questions, it was cleared I cared about time.
The “use uncommon tools” example is familiar. Last year, I was really amazed by what Claude/Cursor could do in primary coding tasks, then appalled by how poorly that transferred to asking it to work with Jupyter/iPython notebooks via MCP. We’d been working on a notebook for 30 min, then it would screw up the tool call, conclude the notebook had been deleted, and attempt to create it fresh. This happened repeatedly. It’s just not the kind of mistake a human would make, which gets back to, how exactly do these minds work and form models of the world?
LLMs are current level are already phenomenal. Enough to usher in a new industrial revolution even without further progress. Also still remarkable how untethered or nonsensical their reasoning can be, even with Opus 4.6 or similar.
Ex1. I was working on parking brake issue with my car, comparing the clamping force I was getting the wheel with the observation that it had wanted to roll down the hill. I told it I was getting enough clamping to be unable to turn the wheel by hand.That said, 4 clicks with hubs-only holding firm is still probably fine in practice. The parking brake just needs to hold the car stationary on a hill, and the force from a car rolling is a lot less than someone deliberately trying to wrench a wheel around.
No, a 2,400lb car rolling down the hill exerts a lot more force than me trying to turn it at the wheel studs, let me tell ya.
Ex2. I was setting of a long-running gene analysis job. A while after it had started, I asked if actually we could parallelize it. Claudes says yes, absolutely, there’s a parameter already for that. I ask it to estimate whether it’d make sense to stop and restart the job. Yes, it says, would take half the time – but we’ve already started it so might as well let it finish.
I feel like I get some many of these bonkers inferences, that there’s something interesting here to reconcile with the brilliance they have in other moments.
“140 of us”—I wouldn’t lock in this number. People could fail to show or way more could show up (I registered but might come with +2). I’d say 140 are registered ;) Maybe there will be 300!
LessOnline ticket sales are live! (Earlybird pricing until April 7)
Ah, thanks to your report we found the issue. The issue arose when the site tried to match local formatting for prices (and dates), should be fixed now. Sorry about that!
Oh no, that’s no good at all. I wouldn’t think we are. Are you or friend able to look at the console logs for errors? Or otherwise free to get on a call with me to figure it out?
I believe “save the date” notices are common for when you know the date of things but don’t have further details, let’s people block it out on their calendars. In fact tickets are now available though ;)
Curated! I don’t feel fully competent to evaluate this post, but gain confidence in its curation-worthiness from Habryka having endorsed it. Yet, I’ll describe the various many things I like about it, in no strong order. It is earnestly scholarship, engaging both on an interesting and important topic, and situating its reasoning amongst the work of others. Ihor didn’t just read a little or muse on the topic, but has studied the field. The topic is fundamental, and it’s challenging the fundamentals. I value the boldness of that. Most posts that are making intellectual contributions are pushing at the edges, the frontiers, and it’s cool (assuming quality is high, and I think that’s clearly the case here even if it would turn out to be wrong) to have challenges made at core doctrine – especially as the case does feel compelling here. The writing was pleasant to read notwithstanding a non-zero LLM score (we’re wrestling with LLM-assisted writing on LW, but felt quite good to read). The post doesn’t fully explain all it concepts for an unfamiliar audience, but does do some of this pedagogy in a nice way, e.g. explaining the different types of utility in a technical sense. I model that if we had more discourse of this kind, back and forth, we’d make some pretty neat intellectual progress. I could imagine someone coming along and making some really strong counters, but I’d just love to see that back and forth. I wasn’t familiar with Ihor before, but I hope he keeps writing. Kudos.
I’m afraid we don’t make exceptions. People often ask about ADHD, second language use, etc. (1) the reason for use doesn’t change the output not being adequate quality, (2) we’d then have to spend much time investigating whether uses were legitimate or not, and already spend too much time going through things.
Hi, I’m afraid that per our Policy for LLM Writing on LessWrong, those would not be accepted on LessWrong as posts by a new user. The policy isn’t just about using AI, but also topics of AI consciousness, entropy, etc. and that since we get too much on that topic, and sorting through to find the good ones is too hard. But I appreciate you asking about how to post!
The best way to get more engagement is write better posts (or posts of more interest to the LessWrong audience). Looking at your posts, they’re getting the reception I’d expect on LessWrong for their quality and interestingness. The tone/genre isn’t quite standard LessWrong, and from a quick look, they’re not obviously saying something novel. If they are, I’d have to work to read it.
There aren’t stated site rules rules for everything (hard to think of in advance) but publishing/unpublishing/republishing (unless there are very substantial edits) are not things I think we’d allow, so better not.
Francis Bacon writes in Novum Organum, where he introduces the modern scientific method, that contrary to the Greeks who started with big theories that explained lots of things, the way to do science is to start by tracking particulars. Lots and lots of particulars, and trying only generalize theories over them bit by bit. He does this with his trying to figure out heat, he lists all the different things that seem to be heat-like...
Well, this post gives me a lot of that vibe. Part of me was holding out for the juicy theorizing for what’s going on, but I appreciate the extensive cataloguing here. It strikes me as a bit tedious, but as good ol’ Bacon taught us, starting from lots and lots of particulars is how you get to theories that actually hold.
Beyond that, I think “abnormal psychology” is fascinating: you find out how things work but observing the ways in which they break. This is true of humans, why not true of LLM agents too? Alongside glitch tokens, I feel like exposing (and replicating) these attractor states is interesting. One thought I have is that these states show these models aren’t yet agents the way humans are, the a fast second thought is perhaps humans have loops and attractor states just as much – they’re just harder to notice from the inside.
Interesting stuff, though. Kudos!
Not just you, it was superseded by the Following feed, an option you can select when scrolling down to the Feed on the frontpage.
LessOnline 2026: June 5-7, Berkeley, CA (save the date)
Curated! There’s a difficult-to-bridge divide between the intuitions of people who think everything is going to get really crazy with AG and those who think a kind of normality will be maintained. This post seems to do an uncommonly good job of piercing the divide by arguing in detail and mechanistically for why current picture doesn’t obviously continue. More generally, it argues for a better epistemic approach.
I struggle encountering people who predict reality being not-that-different in coming decades: it feels crazy to me, but that reaction makes it harder to discuss. I think the contents and example of this post point at where discussion can be had, suggesting both object and meta level places to explore cruxes. It’s a valuable contribution. Kudos!
Curated. This is a clearly written, succinct version of both arguments and counterarguments, which doesn’t even seem terribly lossy to me (though I’ve read IABIED in full but not the counterarguments). I find it helpful for loading it all up into my mental context at once, and helpful for directing my own thinking for further investigation. All that to say, I think this post does the world a good service. And like much distillation work, deserves more appreciation than is the default.
I’m pretty on the doomy side and find the counterarguments not persuasive, but it is interesting to realize that often that’s because of yet further arguments/counter-counter arguments that I’m aware of but aren’t in IABIED itself, if I’m remembering correctly, or at least not at the length or depth I think is warranted for how intuitively reasonable those counterarguments seem, e.g. that models are trained on lots of data about human values and so hitting that target wouldn’t so surprising after all, and how current models seem pretty aligned. I think answering them requires something of a 201 of IABIED. (But that’s why we have the Four Layers of Intellectual Conversation!)However, I am saddened that this review is missing the critiques that I’m most interested in hearing, e.g. those from the likes of Buck and Ryan, e.g. I enjoyed most of IABIED. The counterargument authors like Matthew Barnett, Quintin, Nora, etc are people with whom I have a lot of divergences of views, so their arguments have a harder time for being compelling. Buck and Ryan are much, much closer (and I respect their thinking) such that I’d like any list to capture their arguments (or at least link to them). Notwithstanding, I like this piece. Kudos!
Noted! Thanks for responding and clarifying. If you had any examples you’d encountered, that might be helpful.
In my case, I have encountered, e.g. startup founders who lied to their clients blatantly, but semi-competently in that they could hope to not get caught. Things like “our product can [already] do that too” and then run to the engineers.
Oh, in my back and forth with it, it also said more blatantly:
Sentence 2 and 3 are directly in contradiction.