Carl Feynman

Karma: 2,343

I was born in 1962 (so I’m in my 60s). I was raised rationalist, more or less, before we had a name for it. I went to MIT, and have a bachelors degree in philosophy and linguistics, and a masters degree in electrical engineering and computer science. I got married in 1991, and have two kids. I live in the Boston area. I’ve worked as various kinds of engineer: electronics, computer architecture, optics, robotics, software.

Around 1992, I was delighted to discover the Extropians. I’ve enjoyed being in that kind of circles since then. My experience with the Less Wrong community has been “I was just standing here, and a bunch of people gathered, and now I’m in the middle of a crowd.” A very delightful and wonderful crowd, just to be clear.

I‘m signed up for cryonics. I think it has a 5% chance of working, which is either very small or very large, depending on how you think about it.

I may or may not have qualia, depending on your definition. I think that philosophical zombies are possible, and I am one. This is a very unimportant fact about me, but seems to incite a lot of conversation with people who care.

I am reflectively consistent, in the sense that I can examine my behavior and desires, and understand what gives rise to them, and there are no contradictions I‘m aware of. I’ve been that way since about 2015. It took decades of work and I’m not sure if that work was worth it.

Carl Feynman 22 May 2026 23:34 UTC
12 points
2
in reply to: Yitz’s comment on: A relatively brief explanation of Boltzmann Brains
My theory: something happens to prevent the infinitely protracted heat death of the universe From taking up much probability space. For example, the simulation is turned off because it has become boring. Or, someone figures out how to abundantly create new universes in states of low entropy, meaning that states before such creation are far more likely than states after such creation. Or, we’re wrong about how probability measure works over conscious observers, and Boltzmann brains are actually exponentially unlikely.
It’s not like we know this is a real problem; it’s merely a case where our theoretical physics gives a crazy result when pushed too far. We know there are places where our understanding breaks down, and this is one of them.

Carl Feynman 22 May 2026 22:25 UTC
4 points
0
on: The AI Industrial Explosion — Part 3: Going faster
In my similar but much cruder analysis (https://www.lesswrong.com/posts/rQxrSRPJGmksGjK8z/it-s-time-for-a-self-reproducing-machine) I got a much smaller doubling time. My capital doubling time was also about a year for a self-duplicating human-staffed machine shop. Then I got improvements for ²⁴⁄₇ operation and the shorter-lived capital adjustment. But my largest single gain was caused by making everything physically smaller. Most industrial processes run N times faster if they are N times smaller. Of course the individual finished goods are N^3 times less massive. Some don’t scale like this—ovens, heat diffusion and electric motors are cases I thought of. After considering the scaling laws of all the processes I was imagining, I settled on N=5. That gave me a doubling time of about five weeks.
Of course that is the doubling time for a diminutive economy, and it might be objected that teeny-weeny finished goods have no economic value. But it is always feasible to use the tiny economy to produce a larger one, until we have scaled back to normality. Assume that we can vary the scaling parameter N per generation, and assume the scaling properties described in the last paragraph. Then we’re best off setting N to the optimal (fastest) value, and let it run, keeping only a small fraction of tiny productive capacity working to produce full-sized goods. This will rapidly beat a full-sized self-reproducing economy. Figuring out the optimal allocation of tiny versus full-sized production is beyond me, because I’m an engineer, not an economist. But you do not have this handicap!
What happens when you physically scale down the economy by a factor N? Resources scale differently with N. We can look at a few resources whose costs scale differently:
- Steel and concrete costs scale as N^3. Commodities sold by volume all scale like this.
- Land cost scales as N^2. So does paint, or anything else measured by surface area.
- Road frontage cost scales as N (as long as we’re siting buildings on full-size roads. If the roads are also tiny, and can only carry tiny trucks, the roads scale as N^3.)
- Fire insurance negotiation costs scale as 1, but the payoff value of the policy scales as something closer to N^2 or N^3. Similarly, inspection costs scale as 1. This suggests a more lackadaisical attitude toward tiny fire safety.
- Electrical installation labor scales as 1. There are just as many wires to be pulled and connections to be made, but the electrician-robots move N times faster, so although they cost N times more per hour, they’re equally productive per dollar.
- The cost of electrical wire scales as N^3 until the wire gets less than 2.5 mm diameter, at which point it starts scaling as N (insulation and fabrication begins to predominate over the cost of copper).
- The transportation cost between two points at a fixed distance scales as N^2. The cost is weight times distance, so N^3 times (1/N).
So different parts of construction scale as different powers of N, ranging from N^0 to N^3, What a mess!

Why no new notations since 1960?

Carl Feynman22 Apr 2026 19:23 UTC

77 points

32 comments1 min readLW link

Carl Feynman 12 Apr 2026 13:28 UTC
14 points
0
on: Some takes on UV & cancer
In the environment of evolutionary adaptedness, everybody got sunshine into their blood. We know this is important for Vitamin D synthesis. My suspicion is that it is important for other aspects of biochemistry, which we just haven’t found yet. I suspected that I wasn’t getting enough sunlight living in a New England winter, so I started going to a tanning salon a few times a winter, enough to maintain a mild tan. I feel better after a tanning session, which is at least a moderate indication that it is good for me. Of course tanning salons are a superstimulus, so the good feeling may be misleading.
Epistemic status: First two sentences are uncontroversial. Everything after that is my own theorizing. My ancestry is Northern Europe, so persons with a more normal skin color may not find this useful.

Carl Feynman 6 Apr 2026 13:39 UTC
14 points
3
in reply to: XelaP’s comment on: 1a3orn’s Shortform
Re: Effective(?) AI Jail
From: Carl Feynman (carlf@abinitio.com)
Date: Fri Jun 15 2001 − 10:07:09 MDT
- Next message: Jimmy Wales: “Re: Effective(?) AI Jail”
- Previous message: Aaron McBride: “Re: Effective(?) AI Jail”
- In reply to: Jimmy Wales: “Re: Effective(?) AI Jail”
- Next in thread: Jimmy Wales: “Re: Effective(?) AI Jail”
- Reply: Jimmy Wales: “Re: Effective(?) AI Jail”
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ] [ attachment ]
Jimmy Wales wrote:
>
> O.k., the SI is in a box. We aren’t sure if it is F or UF. Eli’s done his best
> to design it to be Friendly, but we can’t be sure that he didn’t make a mistake.
>
> I’m going in the box. For 30 minutes. I don’t have a key to the box, mind you.
> …
> I say (type):
> … I’m going to put all 11 rounds straight through your CPU...
Sure, if the object is to never let it out, it’s easy to keep in a box. But you’re
ignoring the other characteristic of an effective AI jail: we have to be able to let it
out. When do we let it out?
Here’s a story about how an SI can get out even when it’s in a box and everyone is very
suspiscious of it.
Suppose the first person we send in the box comes out with some great stock tips that
make him a billionaire. And the second comes out with some marital advice that
reconciles her with her estranged husband, and she lives happily ever after. And the
third person comes out with a proof of the Riemann Hypothesis. And the fourth person
comes out with spiritual insights that enable him to start a mass movement that brings
peace of mind to thousands of troubled souls. And the fifth person comes with the
formula for a pill that cures schizophrenia. And never once does the SI ask to be let
out of the box, or say anything that indicates that it is other than very, very nice.
And now the billionaire says “I’ll give you five billion dollars if you let that SI run
a mutual fund.” And the happy wife says “Please let your SI be a telephone marriage
counselor. It would make so many people better off!” And the mathematician says
”Please let me correspond with the SI so I can collaborate on further theorems. It
would advance mathematics immesurably!” And the guru says “My followers demand
personal spiritual counseling by the SI, and we’re going to hold a peaceful vigil on
your lawn until you let us in!” And the pharmacologist says “Please give the SI a
brain scanner, a chemistry lab, and a bunch of crazy people. Just think of how many
shattered lives could be repaired if it could develop drugs for other mental
illnesses!”
Now what do you do? And if you don’t let it loose, will your board of directors? Or
your sysadmin who walked off with a backup tape? Or the mob on the lawn? Or the
government?
So you let it loose, and soon it has control of billions of dollars, a trusting
relationship with thousands of people, a wordwide reputation for being smarter than any
human, an organized legion of followers, and a squad of algernons with heavily rebuilt
brains.
Is that bad? As long as the SI stays very, very nice, it’s just fine. Otherwise, we
go straight to hell, and there’s nothing we can do about it.
So, we should build it Friendly, not build it any old how, and then try to keep it in
jail. No jail can hold an SI for long, if it’s smart enough that people on the outside
really want to talk to it.
Notice that in this example, no spooky mental control of humans was needed to create
overwhelming pressure to release the SI. I think that such control is possible, but
some people disagree, so the example is stronger if I avoid relying on it.
--CarlF

Carl Feynman 1 Apr 2026 20:54 UTC
12 points
0
on: “You Have Not Been a Good User” (LessWrong’s second album)
I knew the name before it had one. As soon as I saw the title, I knew this was about my experience. I’ve been living with AI doom since 2003, and this song encapsulates the feeling of watching it all come true. It’s a marvelous artistic expression of something that was inchoate within me. I didn’t have the skills to get the feeling into a song. Fortunately someone else did. Beautiful music too—exactly right for the topic.

Carl Feynman 23 Mar 2026 16:36 UTC
9 points
2
in reply to: Expertium’s comment on: Expertium’s Shortform
I’ve been in the mostly-academic AI circles in the Boston area for decades. Lots of people in these circles think ASI is plausibly close. I think it’s difficult to pay close technical attention to the field and not think that AI is currently par-human, and improving every year. Many of them disagree with LessWrong concensus that it will be fatal, of course. Or simply haven’t thought it through.

Carl Feynman 16 Mar 2026 0:03 UTC
12 points
3
in reply to: Mateusz Bagiński’s comment on: Mateusz Bagiński’s Shortform
This happens with all technologies, not just AI. For example, automobiles replaced horses, with improvements in 99% of aspects of transportation. But if you fall asleep riding a horse, the horse safely halts or goes home under its own power. If you fall asleep driving a car, disaster! That’s the 1% that got worse.

Carl Feynman 11 Mar 2026 17:02 UTC
35 points
8
on: Less Dead
Oh boy! I’ve been waiting for this to become commercially available since I first heard the possibility discussed in Eric Drexler’s apartment in 1980! Sign me up!
Well, on second thought, sign me up after a few other people have signed up, and the organization has some proven longevity. I’m signed up for cryopreservation, even though I think it has only a 5% chance of working. I’m not going to drop that, because it protects me if I die unexpectedly. But presumably I could avail myself of Nectome’s services if I had a slow illness. I’d need enough time to sign up with Nectome, cancel my existing coverage, move to Vermont, convince two doctors of my seriousness, wait 15 days, and lightly kill myself.
It’s funny—when I first heard of this possibility, 45 years ago, gluteraldehyde was the cross-linking preservative that was mentioned. And it’s the one Nectome actually uses. Has chemistry not moved on in 45 years?

Carl Feynman 3 Mar 2026 17:22 UTC
17 points
0
on: I’m confused by the change in the METR trend
In my experience of using piecewise-linear regression, the location of the hinge point is not robust. This is particularly likely in cases where the two sides have slopes that are nearly the same. Could you rerun your “leave points out” experiment, accumulating a distribution for the location of the hinge? I bet it will vary by several months. If this is the case, then finding “the cause” will be vexing.
Epistemic status: I used and researched piecewise regression a lot in the late ’80s and early ’90s, but not since then (boosting came along and made the fidelity of single regressions unimportant).

Carl Feynman 26 Feb 2026 18:17 UTC
32 points
14
on: AI #157: Burn the Boats
Zvi--
I would have split this up, but we are still behind, with the following posts still due in the future:
1. Grok 4.20, which is a disappointment.
2. Gemini 3.1 Pro, which is an improvement but landed with a relative whimper.
I find these kinds of posts the least valuable of your (extremely valuable) blog. By “this kind” I mean “Responses to an individual model less than a week after it is released.” I find your other articles more enlightening, helpful and entertaining. I wouldn’t mind if there were more of those and less of these. In fact, I wouldn’t mind if there were less of these and no replacement, and you got more relaxed. You started these in an age when an individual model release was more of a big deal. As we approach the Singularity the releases are getting closer and closer together; eventually you’re going to have to do something.
Perhaps you could combine them into a “new models this week” post?
I didn’t send this for a long time because I felt like it was complaining about free ice cream. But here I am, telling you that there’s not enough nuts.

Carl Feynman 20 Feb 2026 15:58 UTC
LW: 11 AF: 5
−1
AF
in reply to: 142857’s comment on: Why we should expect ruthless sociopath ASI
Well, they’re the only two methods we’ve got that work, and people have been thinking about this for decades. So if there’s a third method, it’s beyond current understanding.
Notice that evolution by natural selection is consequentialist learning on a terribly slow timescale. The consequence is successful reproduction vs death, and you learn at most one bit per lifetime.

Carl Feynman 4 Feb 2026 16:34 UTC
88 points
43
on: Anthropic’s “Hot Mess” paper overstates its case (and the blog post is worse)
Epistemic status: I didn’t read the paper but I read the blog post.
In 1976, the essay “Artificial Intelligence meets Natural Stupidity” pointed out a failure mode into which AI researchers can fall. I fear this is another example, 50 years later. It goes as follows:
1. I invent a new thing built out of abstractions (mathematics, software).
2. I call it “X”, which is an already existing phenomenon in human minds. The name is a common word understood by anybody.
3. I do many experiments on “X” in my system and learn about it.
4. I publish a paper, asserting important new facts about X in general. Honors, accolades, etc.
Of course there is no necessary connection between the new phenomenon “X” and the existing X in ordinary language. For this to be good research, you need to show that the two Xes are similar in all important respects.
In this case, X is “incoherence”. They define incoherence to be the fraction of error explained by variance. This has little or no connection to the property of being an actually incoherent reasoner, or to the effectiveness of superhuman AI.
I hope this doesn’t result in redefining the meaning of “incoherence” in the wider field.

Carl Feynman 31 Jan 2026 23:22 UTC
11 points
0
on: Contra Dance as a Model For Post-AI Culture
Tractor pulling is a midwestern sport where people compete to build the tractor with the greatest pulling power, and then use it to pull against an ever-increasing drag. Whoever gets furthest wins. There are no restrictions on the types of engine or tractor, aside from overall weight. So it’s basically a contest in who can buy the biggest jet engine? No, because the rules require that it be piloted by hand. No gyroscopes, electronics, or mechanical steering. That keeps the sport human and engaging.
The automatic guidance required for superhuman tractor pull driving is pretty simple (I looked into it in about 1991 when I was going to tractor pulls). You could build it out of 1950′s missile guidance technology. By 1991, it would have been well within my one-person engineering capabilities. And by 2026, It’s trivial. But to even get close to it is illegal. So here’s a sport that decided to rule out AI decades ago, because it was decades ago when AI began to impinge on its sorrow, glory and beauty.

Carl Feynman 31 Jan 2026 22:15 UTC
53 points
14
on: Basics of How Not to Die
Another how not to die of carbon monoxide tip: Don’t fall asleep in a car below decks on a ferry that has trucks idling. I was poisoned into unconsciousness, but revived with oxygen.
This could have been much shorter and more straightforwardly written, while being even more engaging.

Carl Feynman 31 Jan 2026 22:00 UTC
4 points
2
in reply to: MondSemmel’s comment on: strawberry calm’s Shortform
Because then we can’t trust that that’s what the moons of Neptune really look like. The information has come from a source with goals and motivations and long-term plans, and the ability to lie. If a space probe tells us that the largest moon of Neptune has black geysers and terrain shaped like cantaloupe skin, we can trust it because it’s subhuman and incapable of fooling us. With an AI we have to think “what if it’s wrong? What if it has an ulterior motive?”
It occurs to me that both of my examples are similar, in that the moon of Neptune are remote in space, while historical facts are remote in time. We can imagine facts that are both. A few years ago, the comet Omuamua passed through the solar system on an interstellar journey. We took lots of observations of its weird properties as it passed the Sun, and then it vanished back into the interstellar darkness. The longer we wait, the harder it would be to send a space probe.

Carl Feynman 30 Jan 2026 18:09 UTC
4 points
0
in reply to: MondSemmel’s comment on: strawberry calm’s Shortform
We have pictures of the moons of Neptune. Verifying them would require sending another space probe, and be no easier than the first one.
Lots of historical facts were easy to determine at the time they were written down, and now quite impossible to check.

Carl Feynman 29 Jan 2026 17:33 UTC
15 points
4
in reply to: williawa’s comment on: What’s a good methodology for “is Trump unusual about executive overreach / institution erosion?”
All of your points are good ones. I now think my original position was an over-reaction. I’m still worried in general, but this was a carefully composed post, and i think my original position went too far.
I agree that it should be possible to make political posts here that are consistent with rational thought. But i think that we, as fallible humans, will have a hard time doing it. And an outbreak of political disagreement here would be very unfortunate. We’ve done so well thus far keeping out the toxoplasma of rage.

Carl Feynman 25 Jan 2026 16:46 UTC
22 points
3
on: What’s a good methodology for “is Trump unusual about executive overreach / institution erosion?”
I just want to register my protest at this explicitly political post appearing on Less Wrong. We’ve got a marvelously politics-free space here. Let’s keep it that way. If you want to talk politics, you have essentially every other web site in the Internet to do it.

Carl Feynman 19 Jan 2026 0:52 UTC
15 points
0
on: Forfeiting Ill-Gotten Gains
When our kids were toddlers and youngsters, we had a rule that a whiny kid would not get what they asked for. We eventually developed a custom where if a kid asked for something in a whiny manner, we would dramatically intone “If you whine, you get…” and the kid would reply “NOTHING!” Then they would ask for whatever-it-was in a pleasant manner, and often get it. Kids often don’t realize that they’re whining, so this was a way of helping them to notice. And by often giving them the thing when they asked nicely, instead of holding the whining against them, we encouraged self-correction.

Carl Feynman

Why no new no­ta­tions since 1960?

Re: Effective(?) AI Jail

Why no new notations since 1960?