Yair Halberstadt

Karma: 4,267

Yair Halberstadt 22 Jun 2026 7:16 UTC
7 points
0
on: Coup is the Pareto-optimal social game
It’s a fun game, I own it and play it frequently. Main disadvantages:
1. There are people who dislike lying, and/or social deduction games. This obviously won’t work for them.
2. Winner is often random, there’s not a strong correlation between skill and final outcome.
3. Frequent kingmaker dynamics, where one person can’t win but can decide which of two other people will win.
4. People who are out can be sitting on the side with nothing to do for a few minutes.

Yair Halberstadt 21 Jun 2026 16:57 UTC
7 points
1
on: Policy changes should be rolled out gradually
Putting this in a comment since I didn’t want to address it in the main body of the post:

The main reason this is unlikely to happen is election cycles. Governments have a short time period in which to make their mark, and if the impacts of all their policy changes are only felt towards the end of their mandate / beginning of next government then it’s going to be difficult for them to win reelection.

This is even worse because ministers/heads of department are often even shorter lived than governments, so they’re under high pressure to prove they’ve done something.

It’s also much easier for the next government along to roll back all the previous governments changes if they aren’t yet applicable nationwide. For that reason there’s a strong incentive to make all policy changes as sticky and hard to rollback as possible.

Yair Halberstadt 19 Jun 2026 9:45 UTC
9 points
2
on: Yair Halberstadt’s Shortform
I’m surprised that private neighbourhoods aren’t more common:

A company buys a large plot of land, and build houses, shopping centres schools etc. they rent out the space to households and businesses. Included in the rental fee is services such as garbage collection, maintenance, etc.

In lieu of a police force they have private security which is empowered to terminate the rental contract for problematic individuals and deny people entry to the neighbourhood.

This should be enormously profitable because the land is worth so much more as a neighbourhood than as a Greenfield.

I also imagine it would be far better run than most cities. Poor services bring down rents, so it’ll make sure the streets are always clean. Good transport is essential for success so it’ll operate private busses to nearby urban centres. Etc.

They don’t sell the buildings because then there’s no incentive to provide services/just becomes an ordinary neighbourhood.

Yair Halberstadt 16 Jun 2026 5:38 UTC
3 points
0
on: A frontier AI company should shut down
It would also be great if countries stopped going to war, governments followed sound economic policy, and we started a Manhattan project to solve aging, but none of those seem very likely.

How concretely do you plan to persuade any of the frontier labs to shut down?

Yair Halberstadt 14 Jun 2026 10:44 UTC
14 points
6
on: Wikipedia’s national flavors—French
I think some examples of this in french Wikipedia would be useful

Yair Halberstadt 12 Jun 2026 14:57 UTC
6 points
4
in reply to: Petropolitan’s comment on: PSA: Almost nobody is directly working on superintelligent alignment
Many branches of engineering were based on initial theoretical breakthroughs followed by engineering trial and error.

We understood how rockets worked on a theoretical level long before we tried to build them. It’s reasonable to assume we would understand how intelligence works before we managed to build it.

Yair Halberstadt 12 Jun 2026 7:35 UTC
29 points
0
on: PSA: Almost nobody is directly working on superintelligent alignment
I think theoretical alignment (as opposed to applied alignment, working with current models) is in a slump right now because it’s hard to see how it slots in to the current LLM paradigm.

There was an assumption 5 years ago that AGI requires fundamental insights into intelligence. There was a hope that understanding intelligence better on a theoretical level could also point us towards how to steer such an AGI on the direction we wanted.

That assumption seems to be false. It turns out you can build human level intelligence without understanding the first thing about intelligence just by throwing enough compute and data at the problem. Sure, architecture is important, but architectural improvements are driven far more by trial and error, and informed by a narrow understanding of how LLMs works, than by a grand unified theory of intelligence.

In such a world it’s difficult to see how abstract alignment work slots in. Instead we develop alignment the same way as we develop intelligence: trial and error driven by a narrow understanding of LLMs and current architectures rather than a grand unified theory of alignment.

In that world, good alignment research asks narrow questions like: “can we tell whether an LLM is going off the rails just by monitoring it’s COT”, rather than broad questions like “how do we know whether an LLMs utility function exactly matches humanities coherent extrapolated volition”.

Yair Halberstadt 10 Jun 2026 16:45 UTC
2 points
0
in reply to: Abhishaike Mahajan’s comment on: How to build a cancer vaccine, and whether they will work this time
Thanks, yes I enjoyed that article

Yair Halberstadt 10 Jun 2026 3:05 UTC
7 points
3
in reply to: Luise Woehlke’s comment on: 5 Things I Learned About People From Doing Stand-Up Comedy
I’m ok with context, which summarises the link. Less ok with a bait and switch.

Yair Halberstadt 9 Jun 2026 17:02 UTC
28 points
23
on: 5 Things I Learned About People From Doing Stand-Up Comedy
Hi this seemed interesting, but I’m not looking for less wrong to become a site with tantalising snippets which require you to finish off on a different site, so down voted for that reason.

Yair Halberstadt 9 Jun 2026 11:38 UTC
2 points
0
on: How to build a cancer vaccine, and whether they will work this time
Thank you for this extremely interesting article!

I’m interested if you have an overview of other modern cancer therapies that are being worked on, and whether you think they’re promising?

Yair Halberstadt 7 Jun 2026 17:31 UTC
2 points
0
in reply to: core’s comment on: Why Software Automation Is Hard
They can. But yes, it doesn’t work great. Adjust the metaphor as necessary

Yair Halberstadt 7 Jun 2026 16:48 UTC
3 points
0
in reply to: core’s comment on: Why Software Automation Is Hard
Imagine that you forgot everything about your current company completely between days, not just current task. Every day is like your first day on the job (coming as an experienced dev from different companies). But you can store 1 million words of notes. The notes aren’t carefully curated—they’re just the last million words you happened to write in your notes, and you take notes on everything. Do you still think you’d make decent progress?

Yair Halberstadt 7 Jun 2026 6:05 UTC
6 points
0
on: Why Software Automation Is Hard
I’ve been wanting to write a very similar piece for a while, and you’ve done a far better job than I would have.

Yair Halberstadt 2 Jun 2026 16:49 UTC
3 points
0
in reply to: harsimony’s comment on: Tech I’m skeptical of and why
I would think of it as: as the raw material becomes a greater percentage of the total cost, decreasing total cost becomes harder, both in relative terms (because it’s impossible to halve total cost if raw material is already the majority of the cost) and in absolute terms (because if any decrease in system complexity causes the raw material to be used less efficiently, that will erode savings from the decreased complexity).

In that sense the idiot index acts like pressure making it harder to squeeze costs when they’re low but with very little impact when they’re high.

Yair Halberstadt 2 Jun 2026 9:51 UTC
5 points
0
on: Tech I’m skeptical of and why
It doesn’t seem to me that the idiot index can be used to predict prices the way you’re using:

For example, currently you’re using an estimate of an idiot index of 100 for fusion, and fuel costs at $4K/kg for Deuterium.

Lets say Deuterium was twice as cheap or expensive. Would your estimate for fusion costs halve or double? If so why? The cost of Deuterium provides almost no evidence of the cost of the complicated parts of building a fusion reactor.

I agree the idiot index is useful for establishing a minimum threshold for cost, and is a useful indication for which processes have a lot of potential savings with further optimisation and those that don’t. But I don’t see any justification for using it to predict future costs.

Yair Halberstadt 29 May 2026 10:52 UTC
2 points
0
in reply to: Daniel Tan’s comment on: Daniel Tan’s Shortform
Something that feels directionally correct to me:

If given a query at the end of a long context, and the same query along with just a summary of all relevant information to the query in that context, how different are the responses?

In general we don’t want extraneous details in the context to impact the response to a new query

Yair Halberstadt 29 May 2026 9:07 UTC
7 points
0
in reply to: kbear’s comment on: Trees are mostly made of air and a generalizable lesson for AI safety
Very little, but realising that implies you have a gears level understanding of what’s actually happening in biology rather than just being able to solve the problems you get asked in exams.

Reminds me of https://calteches.library.caltech.edu/46/2/LatinAmerica.htm

Yair Halberstadt 27 May 2026 6:28 UTC
2 points
0
in reply to: leogao’s comment on: leogao’s Shortform
In that case I think it depends what the alternative is: if a sort of parody of psychopathy and self interest where we’d push someone onto a railway track if they were in our way and that was quicker than moving them away, then obviously that would be bad.

If more myopic: people still lend some salt to a neighbour asks without running explicit cost-benefit calculations, and don’t like stealing or hurting people for the most part, just they won’t give up their jobs to run a charity, or risk their lives to go to war, or become doctors for a less than competitive salary, then I think overall the world probably ends up better off.

Yair Halberstadt 27 May 2026 4:34 UTC
3 points
0
in reply to: leogao’s comment on: leogao’s Shortform
What about crusades, Jihads, revolutions, genocides, political oppression, wars of nationalist conquest, etc.?