Clearly virtue isn’t the thing that actually optimizes for success with people. (Unless you specifically want to be successful with people who really care about virtue and are good at judging it)
Amalthea
Karma: 842
This wouldn’t have solved the problem if people hadn’t stopped having children.
I used to think about this all the time.
I remember reading some quote of an observant contemporary of Malthus, who predicted that there may be hope in the most well-off + educated people not choosing to reproduce as much.
That aside, I still think the current situation is extremely surprising, and that unless you had a model that takes the effect of educated people having less children into account in an highly calibrated way, predicting overpopulation was the right call for much of the time. There might have been more evidence in the 1950s though. (And in any case, that doesn’t mean the panic around it was justified)
Tsimerman may be more AGI-pilled than Gowers, it’s probably just that the latter has a more approachable communication style. E.g. https://arxiv.org/abs/2507.09369
If I understand Hendryck’s logic here, then caring 1/1000 as much about a random stranger as about yourself, means you care several million times more about all random strangers combined than about yourself, which you don’t seem to be saying?
Aren’t you dividing twice there, since you:
1) single out a stranger (thus dividing the amount you care about the average stranger by their number)
2) then apply Hendricks central number to that stranger (where now you should be applying the pooled number, since you’re already ignoring all the other group members)
So I think this in fact pretty close to your intuition if interpreted correctly (you say 1e-3, Hendricks says 1e-2).
higher margins rightfully means higher market cap. if your company is barely scraping by, youre not producing as much value.
This should be capturing rather than producing. (Arguably Meta produces negative value).
Wow, kudos—I genuinely wasn’t sure what was going on with the essay since the tone was indeed almost Borges-style (or perhaps a more grounded Bjartur).
I’ll just say that this doesn’t read like Linch.
I agree that based on this we should assume large jumps in capability to be possible (if and when we get said algorithmic progress). I think this doesn’t directly address the ‘scaling will lead to AGI’ claim though: It’s at least plausible that large enough LLMs can be “generally intelligent enough” to outperform humans across the board on general reasoning tasks.
I think it’s a good argument, but Anthropic doesn’t seem quite aligned enough to make it work. E.g. they don’t seem to have been pushing for a coordinated Pause to any real extent (and if they don’t think this would be a good idea, haven’t clarified their position as far as I know).
The study seems to be about what is predicted by experts to be possible, not what is possible afaict?
Is there a mechanism to explicitly run a proposed agreement by the regulator to get their OK?
You’d need the alternative workable approach to not be basically runnable on GPUs, which is maybe plausible, but seems optimistic?
(E.g. anything that can run on a computer would most likely profit quite a bit from the cheap GPU compute even if it’s overall more complex and the current optimizations aren’t as targeted)
I agree with the general concern, but it’d be clearly a move in the right direction on that front?
With this kind of proposal I’m more worried that it could lead to a unilateral slowdown just after having animated China to be much more aggressive on AI.
This is the “stacked S-curves” effect often seen in the maturing of (usually ordinary) technologies. It’s perhaps slightly unusual that it’s more pronounced and “discrete” right now (relatively few innovations leading to large amounts of progress).
The other angles are probably already out there, but haven’t given the chance to shine while the current paradigm can be sufficiently leveraged, so I’m not very hopeful on progress stalling by itself.
We really need the update! I was going to share this with someone who just now has been hit with the full emotional force of realizing what’s going on with AI… but even this right at the beginning doesn’t seem so applicable anymore:
> Some combination of ‘we run out of training data and ways to improve the systems, and AI systems max out at not that much more powerful than current ones’ and ‘turns out there are regulatory and other barriers that prevent AI from impacting that much of life or the economy that much’ could mean that things during our lifetimes turn out to be not that strange. These are definitely world types my model says you should consider plausible.Maybe still plausible but imo less likely at this point than even “all the politicians and world leaders are going to wake up and are going to implement a halfway sensible solution”.
Narration doesn’t work on this.
I think it’s unclear that this was overall bad for Anthropic/Amodei if you factor in the reputational and ideological boost they got (“aura farming” according to roon).
Wouldn’t professors be more likely to only apply if they genuinely care about safety, while younger people (overall) might be still figuring things out and happy for any opportunity to advance their career?