ryan_b

Karma: 5,333

ryan_b 20 Mar 2026 22:30 UTC
3 points
0
in reply to: CharlieT’s comment on: Nullius in Verba: 3rd party evidence for Nectome’s Brain Preservation
Very reasonable position. It would be great to see a post on the legal investigation at some point, if the team can spare the time.

ryan_b 20 Mar 2026 22:23 UTC
6 points
0
in reply to: annafirtree’s comment on: Nullius in Verba: 3rd party evidence for Nectome’s Brain Preservation
These make mechanical sense to me, but I am really more curious about the legal situation behind the decision to pursue the more difficult mechanical process. I suppose the crux of it is I am shocked that right-to-die laws were the pathway through, rather than the pathways made of medical cases where a person was in a coma, declared legally dead, and then woke up when they were unplugged. Or the right to undergo experimental surgeries. Or some law on the books from the space race ensuring it was legal to go into suspended animation so we could colonize another planet.
Which I now suspect is, in fact, illegal.

ryan_b 20 Mar 2026 22:10 UTC
6 points
0
in reply to: CharlieT’s comment on: Nullius in Verba: 3rd party evidence for Nectome’s Brain Preservation
Am I right in inferring that the central problem here is that a lot of places are defining “no heartbeat” as being legally dead, and that is where the problems enter?
Otherwise, why do they even need to be declared dead? Given the full intention of reanimating the person in question, what’s the distinction between this and a medically induced coma? Or the concrete cases of a naturally occurring coma with a person being declared dead but waking up at the last second? Or just a really, really long active medical procedure, leaning on the heart bypass example?
Another relevant edge case: what about that continuous flow total artificial heart company? There’s at least one guy wandering around with no heartbeat, but alive.

ryan_b 20 Mar 2026 11:16 UTC
6 points
0
on: Nullius in Verba: 3rd party evidence for Nectome’s Brain Preservation
What’s the distinction between a pre-mortem process and a Medical Aid In Dying post-mortem one?

ryan_b 25 Feb 2026 17:00 UTC
3 points
0
in reply to: Shiva's Right Foot’s comment on: ricraz’s Shortform
I didn’t know we are allowed to discuss politics here.
Politics in the American political horse-race sense (for non-Americans this means political party competition between Republicans and Democrats) and of the “yay political ingroup, boo political outgroup” in general remains at least taboo, and rightly would be downvoted into oblivion even where it isn’t strictly banned.
But the fully general “politics is the mindkiller” blanket ban on even mentioning politically connected subjects hasn’t been true for years. Among other things, AI is now a highly political subject, but analyzing administrative actions surrounding it, policy consequences, even things like actual political campaigns aimed at legislation (like the vetoed California law) are completely fair game. Direct political science style questions have been popular for years.

At this point, I claim our norms are strong enough that political topics are fine, provided they otherwise follow the norms. I would still support the mods nuking any posts that come pre-mindkilled, of course.

I don’t see it happen often, which is probably a combination of pre-mindkilled people not being attracted to LessWrong, our reputation for being intolerant of it proceeding us, and our mod team being very good.

ryan_b 24 Feb 2026 16:18 UTC
2 points
0
on: AI #156 Part 1: They Do Mean The Effect On Jobs
On the question of automated purchasing of a particular kind of small business: I notice the description of small business owners wouldn’t be out of place as a description of people who are resistant to therapy. In the therapy case the fact that AI is actually an uncaring machine instead of a person is a noted advantage for certain groups and, the claim goes, leads to increased adoption.

This sounds more like AI might be especially good at automating the purchasing of small businesses from owners unenthusiastic about selling.

ryan_b 24 Feb 2026 16:12 UTC
2 points
0
on: AI #156 Part 1: They Do Mean The Effect On Jobs
It is tragic that many, including the architect of this, don’t realize this is bad for liberty.
The Tree of Liberty must be watered from time to time with the blood of patriots, and tyrants. Now if the patriots among you would be so good as to raise your hands...

ryan_b 23 Feb 2026 17:13 UTC
2 points
0
in reply to: TurnTrout’s comment on: TurnTrout’s shortform feed
I have been ruminating about this for a couple of days, and have settled on what feels like a stupid question in order that I should be less stupid:

Why isn’t the capability to distinguish between evals and reality the crux of the problem?

It seems like this ability to distinguish between worlds is at least sufficient for the model to deceptively maintain misaligned goals, even if it isn’t necessary.

ryan_b 10 Feb 2026 17:16 UTC
0 points
0
in reply to: Eli Tyre’s comment on: Stone Age Billionaire Can’t Words Good
I agree on all three counts, but what I am talking about is the rhetorical strategy for trying to communicate the belief that it is very important in the long term to let people keep their stuff, to people who are proposing to take some people’s stuff right now.
I don’t think bad long term consequences would be hard to communicate in this instance (though it would not be easy in the middle of a protest). For example, I expect almost everyone who supports a wealth tax to also oppose the idea of corporate personhood; but these rulings started showing up in the 1800s, and the Citizens United decision showed up in 2010.
There’s a bit of line to walk so as to not misrepresent what OP is believes, but I feel like establishing a link between longer-term bad effects that billionaire tax supporters understand might be as simple as saying “Citizens United was in 2010 and. . .” gestures at things in general

ryan_b 9 Feb 2026 21:56 UTC
31 points
10
on: Stone Age Billionaire Can’t Words Good
that we must stand true to the Code and allow those billionaires to retain the wealth if we want to remain a Just and Wealthy society.
I think this right here is the crux of it. I doubt anyone who supports a wealth tax believes we live in a just society, and I expect them all to believe that the behavior of billionaires actively suppresses our wealth in the sense that you mean it. It looks, to them, like our current crop of billionaires are not honorably wealthy men, but trying very hard to become the new feudal lords themselves.
Protecting the Code is incredibly important NOT because it serves some billionaires, but because it serves every single person in our society that lives above the level of a 16th century peasant.
I think there’s a shorter path to this conclusion that the people supporting the wealth tax will find more understandable, I bet: if we pass a law to take billionaire’s stuff, they’ll use the same law to take our stuff.

ryan_b 9 Feb 2026 19:27 UTC
5 points
0
on: Post-AGI Economics As If Nothing Ever Happens
Excellent post, strong upvote. You’ve done a great job articulating what I felt as basically just a twisting of the guts whenever I read economic analysis of the idea of AGI. Tackling the problem head-on:
Model AI as a firm directly: I believe AI straightforwardly breaks the usefulness of the capital-labor distinction. The central crux for me is the extent to which the AI could perform the knowledge work of corporate management; I claim it doesn’t matter for economic purposes that the source of the decisions is an abstract machine the company owns (or rents), what matters for economic purposes is the level at which the decisions are made. If AI makes the management decisions for the firm, to the rest of the economy they are indistinguishable.
For modelling AI-as-a-firm:

Information Asymmetry: I predict the AI will have an information advantage over most other actors in the economy, and once we cross the AGI threshold over all non-AI actors in the economy, at least eventually. This might be a reasonable economics-view-definition of AGI: the threshold at which it achieves local information asymmetry on all transactions.
Transaction Costs: I expect transaction costs to be systematically lower for AI firms through the time-cost of decisions. I make a concrete analogy to high frequency trading, where a fast trading algorithm can see a new buy order, go purchase the better orders on the market, and return to sell to the original order.
Some second-order items I would like to see:
Principal-Agent problems: This is how economics tackles alignment problems. Currently we model OpenAI/Anthropic as owning ChatGPT/Claude respectively, under capital; if the AIs were instead modeled as firms independently and viewed as subcontractors (albeit with contracts strongly favoring OpenAI/Anthropic) and apply the information asymmetry and transaction cost modifications above, what does a principal-agent model predict?
EMH-breaking threshold: My intuition is that the information asymmetry and transaction cost advantages are mutually reinforcing, but the idea I think is more important is that doing a transaction provides much more detailed information than a price signal. A systematic advantage in completing transactions means a systematic accumulation of higher dimensional information than prices; because the EMH works on price signals, I expect it will be defeated if it is possible to aggregate higher-dimensional signals than price.

ryan_b 29 Sep 2025 22:31 UTC
10 points
2
on: On Dwarkesh Patel’s Podcast With Richard Sutton
It feels to me like Sutton is too deep inside the experiential learning theory. When he says there is no evidence for imitation, this only makes sense if you imagine it strictly according to the RL theory he has in mind. He isn’t applying the theory to anything; he is inside the theory and interpreting everything according his understanding of it.

It did feel like there was a lot of talking past one another when Dwarkesh was clearly talking about the superintelligent behaviors everyone is interested in (doing science, math, and engineering) as his model for intelligence, and Sutton is blowing all of this off only to articulate quite late in the game that his perspective is that human infants are his model for intelligence. If this was cleared up early, it would probably have been more productive.
I have always found the concept of a p-zombie kind of silly, but now I feel like we might really have to investigate the question of an approximate i-zombie: if we have a computer than can output anything an intelligent human can, but we stipulate that the computer is not intelligent....and so on and so forth.

On the flip side, it feels kind of like a waste of time. Who would be persuaded by such a thing?

ryan_b 29 Sep 2025 20:43 UTC
7 points
0
on: I have decided to stop lying to Americans about 9/11
Good job saying a brave thing; on a US-based site with a plurality US membership, this was a risk. Well done.
Out of curiosity, how often do conversations about 9/11 come up? For the most part, we don’t discuss it that much among ourselves except during the anniversary, although I do make note that it was just a couple of weeks ago and indeed the traditional observance is literally just to talk about where we were and what we were doing at the time, which precisely when the observation about cheering would come up.
It may or may not surprise you that while there were basically no rooms cheering in the US, there was a substantial minority population that celebrated after the fact. Mostly these were people who hated finance and globalization (which the twin towers symbolized) or something about foreign policy (imperialism, colonialism, etc) or as some form of divine punishment (tolerating gay people or interracial marriage or what-have-you).
So, thank you for saying your piece. I appreciate the honesty.

ryan_b 24 Sep 2025 20:58 UTC
6 points
1
on: IABIED Review—An Unfortunate Miss
Strong upvote, I appreciate the inside-view context that you have from publishing a similar book. I bought it as a result of this review.

I cannot, alas, promise a side-by-side review. However, there are a couple of questions I am primed to look for, foremost among them right now: how much detail is invested in identifying the target audience? The impression I am getting so far is that it has been approximately defined as not us, but a lot of complaints seem to turn on this question. I see a lot of discussion about laymen but that’s an information level, not a target audience. I don’t know if I have seen much discussion of the target audiences at all outside of the AI policy area, come to think of it.

ryan_b 19 Sep 2025 21:12 UTC
3 points
1
on: ryan_b’s Shortform
The important information I should take from a strong trend is an axis, or a dimension, rather than the default takeaway of a direction.
I have the colloquial habit of talking about a trend as a direction, leaning on the implicit metaphor of physical space for whatever domain in which the trend appears. I’ve only just today started to realize that, while I am pretty sure the physical space (or, probably, geography and maps) metaphor is why I speak that way, there’s no reason not to lean into it as an explicit description within the abstract space for the domain. By this I mean that whatever it is we are talking about (the domain) has several important parts to it (dimensions), and taken together these form the space of the domain.
Returning to the direction v. dimension takeaway, this basically means that the important thing is the dimension (or dimensions) along which the trend moves, so it is worth looking at the opposite direction of the trend as well.

This is basically the same as the idea as taking good advice and reversing it, just looking at changes in the world instead.

ryan_b 26 Aug 2025 22:08 UTC
2 points
0
on: Underdog bias rules everything around me
Boiling this down for myself a bit, I want to frame this as a legibility problem: we can see our own limitations, but outsiders successes are much more visible than their limitations.

ryan_b 25 Aug 2025 20:16 UTC
5 points
0
in reply to: johnswentworth’s comment on: Yudkowsky on “Don’t use p(doom)”
I’m inclined to look at the blunt limitations of bandwidth on this one. The first hurdle is that p(doom) can pass through tweets and shouted conversations in bay area house parties.

ryan_b 25 Aug 2025 19:59 UTC
4 points
2
in reply to: Lukas Finnveden’s comment on: Yudkowsky on “Don’t use p(doom)”
I also think he objects to putting numbers on things, and I also avoid doing it. A concrete example: I explicitly avoid putting numbers on things in LessWrong posts. The reason is straightforward—if a number appears anywhere in the post, about half of the conversation in the comments will be on that number to the exclusion of the point of the post (or the lack of one, etc). So unless numbers are indeed the thing you want to be talking about, in the sense of detailed results of specific computations, they are positively distracting from the rest of the post for the audience.

I focused on the communication aspect in my response, but I should probably also say that I don’t really track what the number is when I actually go to the trouble of computing a prior, personally. The point of generating the number is clarifying the qualitative information, and then the point remains the qualitative information after I got the number; I only really start paying attention to what the number is if it stays consistent enough after doing the generate-a-number move that I recognize it as being basically the same as the last few times. Even then, I am spending most of my effort on the qualitative level directly.
I make an analogy to computer programs: the sheer fact of successfully producing an output without errors weighs much more than whatever the value of the output is. The program remains our central concern, and continuing to improve it using known patterns and good practices for writing code is usually the most effective method. Taking the programming analogy one layer further, there’s a significant chunk of time where you can be extremely confident the output is meaningless; suppose you haven’t even completed what you already know to be minimum requirements, and compile the program anyway, just to test for errors so far. There’s no point in running the program all the way to an output, because you know it would be meaningless. In the programming analogy, a focus on the value of the output is a kind of “premature optimization is the root of all evil” problem.
I do think this probably reflects the fact that Eliezer’s time is mostly spent on poorly understood problems like AI, rather than on stable well-understood domains where working with numbers is a much more reasonable prospect. But it still feels like even in the case where I am trying to learn something that is well-understood, just not by me, trying for a number feels opposite the idea of hugging the query, somehow. Or in virtue language: how does the number cut the enemy?

ryan_b 25 Aug 2025 18:09 UTC
5 points
3
in reply to: Lukas Finnveden’s comment on: Yudkowsky on “Don’t use p(doom)”
I can’t speak for Eliezer, but I can make some short comments about how I am suspicious of thinking in terms of numbers too quickly. I warn you beforehand my thoughts on the subject aren’t very crisp (else, of course, I could put a number on them!)

Mostly I feel like emphasizing the numbers too much fails to respect the process by which we generate them in the first place. When I go as far as putting a number on it, the point is to clarify my beliefs on the subject; it is a summary statistic about my thoughts, not the output of a computation (I mean it technically is, but not a legible computation process we can inspect and/or maybe reverse). The goal of putting a number on it, whatever it may be, is not to manipulate the number with numerical calculations any more than the goal of writing an essay to is grammatically manipulate the concluding sentence, in my view.
Through the summary statistic analogy, I think that I basically disagree with the idea of numbers providing a strong upside in clarity. While I agree that numbers as a format are generally clear, they are only clear as far as that number goes—they communicate very little about the process by which they were reached, which I claim is the key information we want to share.
Consider the arithmetic mean. This number is perfectly clear, insofar as it means there are some numbers which got added together and then divided by how many numbers were summed. Yet this tells us nothing about how many numbers there were, or what the values of the numbers themselves were, or how wide the range of numbers was, or what the possible values were; there are infinitely many variations behind just the mean. It is also true going from no number at all to a mean screens out infinitely many possibilities, and I expect that infinity is substantially larger than the number of possibilities behind any given average. I feel like the crux of my disagreement with the idea of emphasizing numbers is people who endorse them strongly look at the number of possibilities eliminated in the step of going from nothing to an average and think “Look at how much clarity we have gained!” whereas I look at the number of possibilities remaining and think “This is not clear enough to be useful.”
The problem gets worse when numbers are used to communicate. Supposing two people meet in a Bay Area House Party and tell each other their averages. If they both say “seven,” they’ll probably assume they agree, even though it is perfectly possible for the average of what to have literally zero overlap. This is the point at which numbers turn actively misleading, in the literal sense that before they exchanged averages they at least knew they knew nothing, and after exchanging averages they wrongly conclude they agree.
Contrast this with a more practical and realistic case where we might get two different answers on something like probabilities from a data science question. Because it’s a data science question we are already primed to ask questions about the underlying models and the data to see why the numbers are different. We can of course do the same with the example about averages, but in the context of the average even giving the number in the first place is a wasted step because we gain basically nothing until we have the data information (where the sum-of-all-n-divided-by-n is the model). By contrast, in the data science question we can reasonably infer that the models will be broadly similar, and that if they aren’t that information by itself likely points to the cruxes between them. As a consequence, getting the direct numbers is still useful; if two data science sources give very similar answers, they likely do agree very closely.
In sum, we collectively have gigantic uncertainty about the qualitative questions of models and data for whether AI can/will cause human extinction. I claim the true value of quantifying our beliefs, the put-a-number-on-it mental maneuver, is clarifying the qualitative questions. This is also what we really want to be talking about with other people. The trouble is the number we have put on all of this internally is what we communicate but does not contain the process for generating the number, and then the conversation invariably becomes about the numbers, and in my experience this actively obscures the key information we want to exchange.

ryan_b 22 Aug 2025 21:28 UTC
3 points
0
on: DeepSeek v3.1 Is Not Having a Moment
I suspect DeepSeek is unusually vulnerable to the problem of switching hardware because my expectation for their cost advantage fundamentally boils down to having invested a lot of effort in low-level performance optimization to reduce training/inference costs.

Switching the underlying hardware breaks all this work. Further, I don’t expect the Huawei chips to be as easy to optimize as the Nvidia H-series, because the H-series are built mostly the same way as Nvidia has always built them (CUDA), and Huawei’s Ascend is supposed to be a new architecture entirely. Lots of people know CUDA; only Huawei’s people know how the memory subsystem for Ascend works.

If I am right, it looks like they got hurt by bad timing this round same way as they benefited from good timing last round.

Edit: Finally found a reasonable description of what happened. They were programming Nvidia hardware in assembly. My hardware switch guess is confirmed—this has wiped out their primary advantage. If they continue to fade, I think we could fairly assess them as a casualty of politics.