I write software for a living and sometimes write on substack: https://taylorgordonlunt.substack.com/
Taylor G. Lunt
Karma: 698
The same argument that shows a base universe may be computationally richer than our universe (and at least cannot be less computationally rich), also greatly limits the number of simulated universes there could be. The (third branch of the) simulation hypothesis, which posits a very large number of simulated universes ultimately stemming from a single base universe, basically relies on the fact that you do not need at least X bits of information in the base universe to simulate a universe of X bits. If you add that restriction, which I’d say you should, then the whole idea falls apart and the idea you’re living in a simulation is no longer a certainty. At that point, you’re limited to just the regular amount of bits in the base universe for running minds or whatever.
Sam: “Free will is making decisions independent of your neurochemistry, or other physical causes. A decision is made when an answer arrives in consciousness.”
You: “Free will is making decisions when you could have chosen otherwise, if your reasons or circumstances were different. A decision is made after a deliberative process, when you finally utter your choice, or have some feeling of having finally decided.”
I’m not necessarily disagreeing with your definition, but I would guess there is no actual disagreement about the underlying physical world here.
It almost seems like a real, physical disagreement could be whether or not the conscious mind is involved in decision-making, but there’s obviously no way Sam would say something like “consciousness is causally disconnected from the rest of the universe, and can never influence future decision making processes in the brain.” He speaks the way he does because he’s using a different definition of “choice”/”decision” than you are. It’s, again, semantic. It’s not like he would disagree that the process you laid out in the section “Free Will As a Deliberative Algorithm” exists. You both agree about how the brain works, you just don’t agree with what to call things. I don’t agree that any of your “Final Cruxes” are non-semantic in nature. They either come from a difference in definition of the term “free will”, or the terms “choice”/”decision”.
The semantic nature of this debate is revealed when you say things like:
I don’t understand why he seems to place so much importance in ultimate authorship
This means, “Sam is using the term ‘free will’ to mean X, but I’d prefer if he meant Y.” The reason he places so much emphasis on it, by the way, is because some people feel they are able to escape determinism through the raw power of their consciousness, and it’s those people Sam is arguing against by showing that everything in consciousness is the result of some proximate, non-conscious input.
As for which version of the term “free will” we should use, I personally don’t care. I only really hear the term get used in free-will debates anyway.
I agree that it’s plausible there could be some benefit to creating an AI prediction market.
I mostly haven’t taken any of the other AI benchmarks seriously, but I just looked into ForecastBench and surprisingly it seems to me to be worth taking seriously. (The other benchmarks are just like “hey, we promise there aren’t similar problems in the LLM’s training data! Trust us!”) I notice their website suggests ForecastBench is a “proxy for general intelligence”, so it seems like I’m not the only one who thinks forecasting and general intelligence might be related. I agree it’s not super well-defined, but I mean it in the way I assume the ForecastBench people mean it, which is the ability to, like, generally do stuff at a minimum of a human level.
I think I don’t take that chart particularly seriously though. A lot of AI predictions hinge on someone using a ruler to naively extrapolate linear progress into the future, and we just don’t know if that’s what’s going to happen. I’d personally guess it isn’t. Basically because LLMs got some one-time gains by scaling large enough to be trained on the whole Internet. They may continue to scale at the same pace, or they might not. Either way, I don’t think a linear extrapolation is proof they will.
I’m happy to confirm that large language models still regularly make farcical mistakes in January 2026, when using them for novel, real-world problems.
Maybe once a day for me, an LLM makes an extremely elementary mistake. I usually avoid giving examples because they’re coding related and might not make sense if you don’t code (and because it’s easy to nitpick any example), but just yesterday, I was trying to replace a hardcoded SVG in my code with a HeroIcons React component.
The SVG looked like this:
<svg fill="none" strokeWidth="2" stroke="currentColor" viewBox="0 0 24 24" > <path d="(...)"></path> </svg>And the new HeroIcons component looked like this:
<ExclamationTriangleIcon />The HeroIcons component matched almost exactly, except the stroke width was too thin on the new component. I wasn’t familiar with HeroIcons React components, so I asked Claude Opus 4.5 to make them match exactly in terms of thickness etc. Claude swore up and down there was no way to change the stroke of a HeroIcons component, and told me I’d have to “just use the inline SVG”, “accept the difference”, or simply “match by adjusting the inline SVG”. That is, in order to make them match, I should change not the new HeroIcon component, but the old SVG! That’s totally incoherent. After telling Claude that was nonsense, it went on to give me an overly verbose version of the correct answer (which was to use the
stroke-2TailwindCSS class).A human being responding this way would be fireably incompetent. I think we tolerate it from LLMs because we’ve gotten used to it, and because they’re really fast, so if a query doesn’t work out, whatever, make a new one or solve your problem without LLMs. But yes, they are still very stupid, and say stupid stuff to me on a daily basis.
“By accident”, when working with a stochastic black-box model, doesn’t mean much. Aside from that, I’ve already seen moderately clever prose from LLMs tasked to emulate human writers. Not quite enjoyable to read, but, insofar as there’s an objective binary standard for “clever”, I’d say the bar is passed.
I would love to see an example. I saw people saying something similar about AI poetry once, but then I read the poetry and it was trash. The people just didn’t have any taste when it came to poetry, and thought any vague language that rhymed basically constituted an amazing poem.
A mathematician rubber duck debugging his way to a solution for an open mathematical problem with an LLM has happened, but we can’t really say how much or how little of a shared thought process came from the LLM.
I personally won’t give the AI any credit unless it does it by itself. After all, I said AI won’t generate important scientific breakthroughs, not that it couldn’t be used in some way to help generate a breakthrough.
AI still can’t drive a damned car well enough that if I bought a car I wouldn’t have to. Aren’t we already there? I think the issue is mainly regulatory, at this point.
Are we? Even in a blizzard at night? I’m Canadian, so that matters for me.
As for the stuff about the architecture, I thought about it more, and developed more fleshed out ideas about what I think is limited about the architecture in later posts. I think the fact the LLMs don’t update their weights during thinking is pretty limiting (online learning). But I’m essentially expanding on my intuition, and I have less to bring to the table in terms of analyzing the architecture than I do in saying “this doesn’t work as well as people say it does right now, this hasn’t improved as much as people say it has, and the benchmarks are a lie.”
The only cost would be the cost to run the models
If you want good predictions, there is another cost, which is to gather information. AI can no doubt outperform humans eventually using information available on the internet, and that would be great, but the point of my original post was that there’s a limit to how good you can get doing only that, without going out and gathering new information.
As soon as the end of this year, we appear to be heading into an era where forecasting isn’t the domain of humans anymore.
Bots are already outperforming humans on some markets because of speed. And computer programs/AI will continue to grow in their capabilities. But I’d be shocked if AI were generally better than humans at forecasting in the next year or two. Forecasting is predicting, which in the limit requires general intelligence, so I don’t think forecasting falls until everything falls. Though maybe you only disagree because you think everything falls in the next year or two, idk.
The dream is that prediction markets greatly outperform individual experts, but there’s a limit on how much this can actually happen. The reason prediction markets aren’t more useful is that you can only profit from a prediction market if gathering information is cheaper than the money you’d make from gathering it.
Let’s imagine I write down either “horse” or “donkey” on a slip of paper, and put that paper in a drawer in my kitchen. I then create a prediction market based on what’s written on the paper. The market would sit at around 50%. Maybe people would analyze all the public information about me, and find out that I once rode a horse when I was seven, or whatever. So maybe it sits at 51% for horse, and 49% for donkey. And despite a really sophisticated analysis of all the public information, sifting through my social media posts, trying to guess which word I’d write, you really have no idea what’s written on the paper. The 51% horse estimate would be robust, but uninformative about reality.
Now let’s say the prediction market for my slip of paper went really viral, and billions of dollars were being spent on it. Since people really like money, they decide to start breaking into my house to see the paper, or kidnapping me, or whatever. Now the market is at 99% donkey, 1% horse, and some people made a lot of money correcting the market. And now the market is much more informative about what’s on the slip of paper.
Why was the market originally so uninformative? It’s because the cost of acquiring better information (including the risk of going to prison) was higher than the value of that information. Prediction markets can only incentivize the collection of information that’s cheaper to collect than the money you stand to gain from collecting it. It’s only worth launching a satellite into space to view oil tank levels if there’s a huge market for oil. If the market for oil was only $1 million, then you would not spend over a million dollars launching a satellite to give you an edge trading on that market. And hence our public estimates of the world’s oil supply would be worse if the market for oil was much smaller.
We know, based on the Polymarket market, there’s a 22% chance Ali Khamenei will be ousted by January 31. We can trust that’s a good estimate with respect to the public information available. But it’s not very informative. The public information is just not that good. If we could fully analyze every atom on Earth with a far-future supercomputer, we could get the estimate down below 1% or up above 99%. The future is basically knowable. But the prediction market is not a tool that makes the future knowable in this case.
At least, not at a volume of $19.5 million. Maybe if the volume was much higher, people would be willing to hack into Donald Trump’s emails, or for that matter go to Iran and shoot Khamenei themselves. Then the market would collapse, and we’d know the truth. But until such time, no amount of recombination or analysis of the information we have will tell us what’s going to happen.
(These thoughts were inspired by Scott Alexander’s recent post, but not really directly in response to that post.)
Yeah that seems to be the most serious one, and the only one I could see that I had a real issue with.
What claims were fabricated, specifically? It seems like mostly minor stuff. As in, a man with visual agnosia probably did mistake very different objects, like his wife or his hat, though maybe Sacks created that specific situation where he mistook his wife for his hat just for dramatic effect. It’s shitty that he would do that, but I still feel that whatever I believed after reading The Man Who Mistook His Wife for a Hat I was probably right to believe, because the major details are probably true?
Hey, I did Halfhaven, and I’m not sure it’s right to say it’s really a faster pace than Inkhaven, since Inkhaven was an in-person residency where the residents were working either part-time or not at all, and could focus entirely on writing. Halfhaven, on the other hand, was something you did in addition to your normal life.
I kind of agree that one post a day (or every other day) feels too frequent, but also, too frequent for what? Is the goal to produce great posts during the event, or to improve as a writer? I think the optimal frequency for these two goals are likely different. If the goal is to get better at writing quickly, then I’m reminded of the story people quote from the book Art & Fear:
The ceramics teacher announced on opening day that he was dividing the class into two groups. All those on the left side of the studio, he said, would be graded solely on the quantity of work they produced, all those on the right solely on its quality.
His procedure was simple: on the final day of class he would bring in his bathroom scales and weigh the work of the “quantity” group: fifty pound of pots rated an “A”, forty pounds a “B”, and so on. Those being graded on “quality”, however, needed to produce only one pot – albeit a perfect one – to get an “A”.
Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the “quantity” group was busily churning out piles of work – and learning from their mistakes – the “quality” group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.
It may be the case that the optimal pace for learning will simply feel too fast, because it’s not the optimal pace for exploiting your existing skills to make great posts (in the short term).
Thanks! I wasn’t sure anyone was even reading them until the end, but apparently people liked them. Now I wish I’d had a bit more time to read more of the posts to seek out ones I liked, especially in this last one when I was busy and only read a handful.
And thanks about The Confession. I think most of the fiction I write doesn’t fit here on LessWrong so I mostly just write it and then nobody sees it. I’m trying to get better at having protagonists that are actually trying to do something, rather than just ending up in a situation. In general I’d like to have a better understanding of what story is, so later maybe I can bring some storytelling ability back into my nonfiction writing.
We just need to invent a device that reliably shoots you in the head if you experience any suffering above a certain threshold.
I don’t fully buy this alternate explanation, because I find the same increase in literacy when I read older books that were not so popular even when they were published. I’ve even read some letters written by normal people in the ’70s, and they seemed to be shockingly literate for normal people.
That said, I’m sure the effect you’re pointing to is real, I just don’t think it’s the whole story.
I don’t mind if Homer Simpson sutures my wounds, as long as there’s a pathway to me getting in front of a real brilliant person if I have a tricky health problem. The problems start when Homer Simpson starts thinking he’s brilliant and starts blaming you when he can’t figure out what’s wrong with you. And when you can’t tell him apart from the real brilliant people.
I don’t think it’s wildly popular, but it has around 500 reviews on the Canadian Amazon, which seems okay for a reference book, and is similar to the number of reviews for If Anyone Builds It, Everyone Dies. I think they keep making new editions because it’s popular enough. I don’t know how to estimate more precisely than that.
Thanks for the context.
I just wrote a comment defending sneering recently on LessWrong, and I think this is a good time to put my beliefs into action.
You may feel smart, but your comment does not convey any of that intelligence, if you have it. You may be very confident in your opinions, but you are doing nothing to confer that confidence to anyone else. If you want me to believe what you believe, you’ll have to be convincing. A digital temper tantrum won’t do it.
The solution is not in fact to add more flowery prose or complicated sentences, its to write about something else.
I would certainly never suggest this. You seem to be implying that good prose is independent of useful prose, but it’s not. Good prose isn’t just flowery and fancy, it is respectful of the reader and their time, and delivers a message clearly and in an entertaining way.
It seems like you’re acknowledging the 2022 version is a bloated waste of space, while also suggesting at the same time that actually it’s very informative, and that this is all just relative? I definitely do not believe this is all just relative, or that girls forced to read Emily Post in the 1920s would have thought the writing in the book was hollow. It’s clearly full of useful information. Not to mention wit and charm. I weep for the future of our species if women in 2122 would think the 2022 version was “so informative!”
Thanks! I definitely agree, and that’s what I was alluding to when I said more people were entering the intellectual class, possibly diluting its literacy. I learned about this from Paul Fussell’s book Class. He suggested the same number of people, around ten percent, are still going to real university, and the rest are essentially fakes.
I’m not sure what it’s like for American high schools, but in Alberta (Canada) we had different streams for different kids. Dash-1 courses were for academic stream kids, dash-2 was either non-academic or for someone who wants a college certificate or a trade, dash-3 focuses on employability, and dash-4 is for students with learning disabilities. There was also an extra distinction between normal dash-1 students and dash-1 students who were also taking the IB program, which is like AP. I’ve never thought about it, but I’ve never heard of anything like this in America, and I wonder if it’s common or not. It seems like a pretty good idea to keep different groups of students separate. Though I don’t really see a point of having the dash-2 and dash-3 students there at all. Let them join the workforce and make some money. It doesn’t matter how valuable you think the information is for them. They’re not paying attention anyway.
Everyone was just pretending to understand each other? Some kind of social signaling game? Is there a reason this seems plausible to you?
He’s a neuroscientist and a materialist, and I don’t think he’s an epiphenomenalist.
In the excerpts in the OP, he gives an epiphenomenalistic vibe because he’s responding to people who think that free will allows a person to violate the laws of physics (or a person who thinks a lack of free will implies a complete lack of ability to make choices). He says, “You are part of the universe and there is no place for you to stand outside of its causal structure.” He tries to show that consciousness is entirely downstream of physical causes. This does not imply, however, that consciousness is not also upstream of physical effects. Here’s another excerpt where he mentions consciousness is part of a larger causal framework:
(https://podcasts.happyscribe.com/making-sense-with-sam-harris/241-final-thoughts-on-free-will)
This doesn’t sound different from OP’s view, at a physical level.