Carl Feynman

Karma: 1,781

I was born in 1962 (so I’m in my 60s). I was raised rationalist, more or less, before we had a name for it. I went to MIT, and have a bachelors degree in philosophy and linguistics, and a masters degree in electrical engineering and computer science. I got married in 1991, and have two kids. I live in the Boston area. I’ve worked as various kinds of engineer: electronics, computer architecture, optics, robotics, software.

Around 1992, I was delighted to discover the Extropians. I’ve enjoyed being in that kind of circles since then. My experience with the Less Wrong community has been “I was just standing here, and a bunch of people gathered, and now I’m in the middle of a crowd.” A very delightful and wonderful crowd, just to be clear.

I‘m signed up for cryonics. I think it has a 5% chance of working, which is either very small or very large, depending on how you think about it.

I may or may not have qualia, depending on your definition. I think that philosophical zombies are possible, and I am one. This is a very unimportant fact about me, but seems to incite a lot of conversation with people who care.

I am reflectively consistent, in the sense that I can examine my behavior and desires, and understand what gives rise to them, and there are no contradictions I‘m aware of. I’ve been that way since about 2015. It took decades of work and I’m not sure if that work was worth it.

Carl Feynman 5 Oct 2025 0:10 UTC
10 points
2
in reply to: Baybar’s comment on: Checking in on AI-2027
I am amused that we are, with perfect seriousness, discussing the dates for the singularity with a resolution of two weeks. I’m an old guy; I remember when the date for the singularity was “in the twenty first century sometime.” For 50 years, predictions have been getting sharper and sharper. The first time I saw a prediction that discussed time in terms of quarters instead of years, it took my breath away. And that was a couple of years ago now.

Of course it was clear decades ago that as the singularity approached, we have a better and better idea of its timing and contours. It’s neat to see it happen in real life.
(I know “the singularity” is disfavored, vaguely mystical, twentieth century terminology. But I’m using it to express solidarity with my 1992 self, who thought with that word.)

Carl Feynman 4 Oct 2025 23:47 UTC
10 points
5
in reply to: Oliver Sourbut’s comment on: Checking in on AI-2027
Here’s a try at phrasing it with less probability jargon:
The forecast contains a number of steps, all of which are assumed to take our best estimate of their most likely time. But in reality, unless we’re very lucky, some of those steps will be faster than predicted, and some will be slower. The ones that are faster can only be so much faster (because they can’t take no time at all). On the other hand, the ones that are slower can be much slower. So the net effect of this uncertainty probably adds up to a slowdown relative to the prediction.

Does that seem like a fair summary?

Carl Feynman 4 Oct 2025 22:54 UTC
6 points
2
on: Accelerando as a “Slow, Reasonably Nice Takeoff” Story
Some may wonder at the mention of “empire time” in the second excerpt from chapter 5. It refers to a kind of artificially constructed simultaneity available to civilizations which have mastered both traversable wormholes and near-light-speed travel. It doesn’t really do much for a civilization bounded within the orbit of Jupiter, which is only about a light-hour across. I think Stross included it as a flavor phrase. It’s marvelously evocative even if you don’t know what it means.
Back in the early ’90s, when all this singularity stuff was much more theoretical, I remember empire time making a big impression on me. It was neat how we could discern some of the contours of future possible civilizations before we got there.
You can read more about it here: http://www.aleph.se/Trans/Tech/Space-Time/wormholes.html#6

Carl Feynman 27 Aug 2025 20:00 UTC
2 points
0
in reply to: stavros’s comment on: Four ways learning Econ makes people dumber re: future AI
Increasing inequality has been a thing here in the US for a few decades now, but it’s not universal, and it’s not an inevitable consequence of economic growth. Moreover, it does not (in the US) consist of poor people getting poorer and rich people getting richer. It consists of poor people staying poor, or only getting a bit richer, while rich people get a whole lot richer. Thus, it is not demand destroying.
One could imagine this continuing with the advent of AI, or of everyone ending up equally dead, or many other outcomes.

Carl Feynman 27 Aug 2025 19:43 UTC
6 points
0
on: Neuroscience of human sexual attraction triggers (3 hypotheses)
This suggests the perfect date would be to meet at an amusement park, go on a roller coaster together, walk separately to the next roller coaster, and so on.

Carl Feynman 18 May 2025 16:50 UTC
2 points
0
on: How quickly could robots scale up?
I wrote a LessWrong article that tries to estimate doubling time for a self-reproducing robot. A critical step is that smaller robots are faster. Most manufacturing processes scale such that they get N times faster as they get N times smaller. I picked N=4, for reasons explained in the article. I concluded the doubling time is five weeks. So the time to a billion robots is on the order of five years.

Even if your goal is a human-size robot, you’re better off building small robots to build it, since they work faster. I assumed fairly clumsy hardware, but software comparable to a human machinist in cleverness.

Carl Feynman 6 Apr 2025 0:37 UTC
8 points
2
on: Most Questionable Details in ‘AI 2027’
Nitpick: No single organism can destroy the biosphere; at most it can fill its niche & severely disrupt all ecosystems.

Have you read the report on mirror life that came out a few months ago? A mirror bacterium has a niche of “inside any organism that uses carbon-based biochemistry”. At least, it would parasitize all animals, plants, fungi, and the larger Protozoa, and probably kill them. I guess bacteria and viruses would be left. I bet that a reasonably smart superintelligence could figure out a way to get them too.

Carl Feynman 22 Mar 2025 22:47 UTC
4 points
0
in reply to: Daniel Tan’s comment on: Daniel Tan’s Shortform
Quite right. AI safety is moving very quickly and doesn’t have any methods that are well-understood enough to merit a survey article. Those are for things that have a large but scattered literature, with maybe a couple of dozen to a hundred papers that need surveying. That takes a few years to accumulate.

Carl Feynman 22 Mar 2025 16:00 UTC
3 points
0
in reply to: Daniel Tan’s comment on: Daniel Tan’s Shortform
Could you give an example of the sort of distinction you’re pointing at? Because I come to completely the opposite conclusion.

Part of my job is applied mathematics. I’d rather read a paper applying one technique to a variety of problems, than a paper applying a variety of techniques to one problem. Seeing the technique used on several problems lets me understand how and when to apply it. Seeing several techniques on the same problem tells me the best way to solve that particular problem, but I’ll probably never run into that particular problem in my work.
But that’s just me; presumably you want something else out of reading the literature. I would be interested to know what exactly.

Carl Feynman 10 Mar 2025 16:30 UTC
9 points
1
in reply to: ChristianKl’s comment on: So how well is Claude playing Pokémon?
When I say Pokémon-type games, I don’t mean games recounting the adventures of Ash Ketchum and Pikachu. I mean games with a series of obstacles set in a large semi-open world, with things you can carry, a small set of available actions at each point, and a goal of progressing past the obstacles. Such games can be manufactured in unlimited quantities by a program. They can also be “peopled” by simple LLMs, for increased complexity. They don’t actually have to be fun to play or look at, so the design requirements are loose.

There have been attempts at reinforcement learning using unlimited computer-generated games. They haven’t worked that well. I think the key feature that favors Pokémon-like games is that when the player dies or gets stuck, they can go back to the beginning and try again. This rewards trial-and-error learning to get past obstacles, keeping a long-term memory, and to re-plan your approach when something doesn’t work. These are capabilities in which current LLMs are notably lacking.
Another way of saying what Claude’s missing skill is: managing long-term memory. You need to remember important stuff, forget minor stuff, summarize things, and realize when a conclusion in your memory is wrong and needs correction.

Carl Feynman 10 Mar 2025 0:48 UTC
4 points
0
in reply to: ChristianKl’s comment on: So how well is Claude playing Pokémon?
True. I was generalizing it to a system that tries to solve lots of Pokémon-like tasks in various artificial worlds, rather than just expecting it to solve Pokémon over and over. But I didn’t say that, I just imagined in my mind and assumed everyone else would too. Thank you for making it explicit!

Carl Feynman 9 Mar 2025 18:05 UTC
7 points
0
on: A model of the final phase: the current frontier AIs as de facto CEOs of their own companies
This is an important case to think about. I think it is understudied. What separates current AIs from the CEO role? And how long will it take? I see three things:
- Long term thinking, agency, the ability to remember things, not going crazy in an hour or two. It seems to me like this is all the same problem, in the sense that I think one innovation will solve all of them. This has a lot of effort focused on it. I feel like it’s been a known big problem since GPT-4 and Sydney/Bing, 2 ¹⁄₂ years ago. So, by the Lindy principle, it should be another 2 ¹⁄₂ years until it is solved.
- Persuasiveness. I’ve known a few CEOs in my life; they were all more persuasive than average, and one was genuinely unearthly in her ability to convince people of things. LLMs have been steadily increasing in persuasiveness, and are now par-human. So I think scaling will take care of this. Perhaps a year or two?
- Experience. I don’t know how much of this can be inculcated with the training data, and how much of it requires actually managing people and products and thinking about what can go wrong. Every CEO has to deal with individual subordinates, customers, and counterparties. How much of the job is learning the ins and outs of those particular people? Or does it suffice to have spent a thousand subjective years reading about a million people? So we may already have this.
(Side point: As an engineer watching CEOs, I am amazed by their ability to take a few scattered hints spread across their very busy days, and assemble them into a theory of what’s going on and what to do about it. They’re willing to take what I would consider intolerably flimsy evidence and act on it. When this doesn’t work, its called “jumping to conclusions”, when it does work it’s called “a genius move”. AIs should be good at this if you turn up their temperature.)

Carl Feynman 9 Mar 2025 17:15 UTC
14 points
2
on: Phoenix Rising
Eyes are exactly the worst tissue in which to express luciferase. Lighting the eyeball from within prevents seeing things outside the eye, causing blindness. Make your cat glow anywhere else!
Epistemic status: I know you’re kidding. So am I.

Carl Feynman 9 Mar 2025 17:01 UTC
6 points
−5
in reply to: SpectrumDT’s comment on: Why it’s so hard to talk about Consciousness
My charity extends no further than the human race. Once in a while I think about animal ethics and decide that no, I still don’t care enough to make an effort.
A basic commitment of my charity group from the beginning: no money that benefits things other than people. We don’t donate to benefit political groups, organizations, arts, animals, or the natural world. I’m good with that. Members of the group may of course donate elsewhere, and generally do.
We’ve been doing this since 1998, decades before Effective Altruism was a thing. I don’t have a commitment to Effective Altruism the movement, just to altruism which is effective.

Carl Feynman 9 Mar 2025 16:24 UTC
4 points
1
in reply to: Nate Showell’s comment on: What is the best / most proper definition of “Feeling the AGI” there is?
It seems to me that I have done a lot of careful thinking about timelines, and that I also feel the AGI. Why can’t you have a careful understanding what timelines we should expect, and also have an emotional reaction to that? Reasonably coming to the conclusion that many things will change greatly in the next few years deserves a reaction.

Carl Feynman 9 Mar 2025 14:49 UTC
3 points
0
on: So how well is Claude playing Pokémon?
A task like this, at which the AI is lousy but not hopeless, is an excellent feedback signal for RL. It’s also an excellent feedback signal for “grad student descent”: have a human add mechanisms, and see if Claude gets better. This is a very good sign for capabilities, unfortunately.

Carl Feynman 5 Mar 2025 13:37 UTC
5 points
0
in reply to: SpectrumDT’s comment on: Why it’s so hard to talk about Consciousness
You would suppose wrong! My wife and I belong to a group of a couple of dozen people that investigates charities, picks the best ones, and sends them big checks. I used to participate more, but now I outsource all the effort to my wife. I wasn’t contributing much to the choosing process. I just earn the money 🙂.
What does this have to do with my camp#1 intuitions?

Carl Feynman 4 Mar 2025 17:18 UTC
4 points
−1
in reply to: SpectrumDT’s comment on: Why it’s so hard to talk about Consciousness
I don’t think enjoyment and suffering are arbitrary or unimportant. But I do think they’re nebulous. They don’t have crisp, definable, generally agreed-upon properties. We have to deal with them anyway.
I don’t reason about animal ethics; I just follow standard American cultural standards. And I think ethics matters because it helps us live a virtuous and rewarding life.

Is that helpful?

Carl Feynman 4 Mar 2025 2:51 UTC
4 points
0
in reply to: dirk’s comment on: Why it’s so hard to talk about Consciousness
Well, I’m glad you’ve settled the nature of qualia. There’s a discussion downthread, between TAG and Signer, which contains several thousand words of philosophical discussion of qualia. What a pity they didn’t think to look in Wikipedia, which settles the question!
Seriously, I definitely have sensations. I just think some people experience an extra thing on top of sensations, which they think is an indissoluble part of cognition, and which causes them to find some things intuitive that I find incomprehensible.

Carl Feynman 4 Mar 2025 2:31 UTC
5 points
0
in reply to: espoire’s comment on: Why it’s so hard to talk about Consciousness
Well, I can experience happiness and sadness, and all the usual other emotions, so I just always assumed that was enough to make me a morally significant entity.