Agree with the general vibe, disagree with the specific example: I don’t think signposting echo chambers makes you immune to them.
Harjas
Karma: 255
I think it at the very least establishes precedence for pauses / proves that pauses are feasible. “We’ve done it before, let’s do it again but longer” sounds much more convincing than the alternative.
The review mentions it as an afterthought. In its opening section, the author writes:
There’s no redemption here, no moral uplift, no lessons save for perhaps the grimmest and most nihilistic “lesson” I’ve ever encountered in any story, Holocaust-related or otherwise: that when confronted with the unthinkable, most people’s natural tendency is denial.
And when discussing the actual lives saved:
Rudi and Fred do achieve one small victory: in reaction to their report—and to Roosevelt’s warning that the U.S. will punish Nazi collaborators after the war—the Hungarian regent stops deportations long enough to save an estimated 200,000 lives.
200,000 lives is not a small victory to me.
I think the paper answers most of these questions and recommend you read it! It’s short and hard to summarize without losing what gives its arguments force
For sure. I am perpetually skeptical of schadenfreude in myself and others; it just seems entirely orthogonal to “is this thing good on its own merits or not?”, which is what I actually care about. I think this post was interesting and worth making, but I wanted to note my hesitation anyway.
I understand that, I just don’t like the framing of “he can’t complain about some turnaround” and would’ve preferred something like “I think Tyler is making xxx mistake and we can probably learn from it in yyy way.” If someone explicitly acknowledges that something they’re about to do is uncouth, I usually expect them to justify it with a stronger reason. I think the italicized paragraphs at the end (which iirc weren’t there when I first made my comment?) are enough to satisfy my concern.
In air? Papers I’ve dropped, feathers from my clothes. But, most items I drop don’t seem to accelerate that much beyond the initial period of “violent acceleration,” which Aristotle sort of describes but lacks the mathematical language to calculate. I think it’s more or less true that heavy objects fall faster than light objects due to air resistance; if it wasn’t, it would’ve been discovered long before Galileo.
From the paper:
it was already pointed out as early as by Philoponus in the VIth century, that the speed of fall is not proportional to the weight: a ball of lead doesn’t reach ground from a specific height in half the time of ball of half its weight.
There’s a reason why you can sometimes get people with the whole “what’s heavier? A pound of feathers or a pound of bricks?” gag.
(Also consider items in water: basically anything I’ve ever dropped in water either reaches “terminal velocity” or begins to float almost instantly.)
Pychoanalysing others is slightly uncouth, but Tyler started it[1], so I don’t feel bad here.
I don’t even disagree with the points you’re making, I just really don’t like “he started it!” as a justification. If you believe that this post will be good for discourse / the world, you should just say so and write it in that vein; if you don’t think it will be good and wrote it purely to strike back at Tyler, I think you should maybe reconsider whether this was a good idea or not.
(“it is useless to be superior...”)
Counterpoint: Aristotelian physics was mostly right.
Aristotle’s physics is the correct approximation of Newtonian physics in a particular domain, which happens to be the domain where we, humanity, conduct our business. This domain is formed by objects in a spherically symmetric gravitational field (that of the Earth) immersed in a fluid (air or water) and the main celestial bodies visible from Earth.
For a student who has learned physics in a modern school it may sound strange to start physics by studying objects in a fluid. But for somebody who hasn’t it may sound strange not to: everything around us is immersed in a fluid. Aristotle’s physics is a highly nontrivial correct description of these phenomena, without mistakes, and consistent with Newtonian physics, in the same manner in which Newtonian physics is consistent with Einstein physics in its domain of validity.
An interesting point from a post about frontier AI usage in the AI safety community:
Recent frontier AI models have proven quite adept at hijacking people’s stated moral commitments. I’ve seen this play out in two ways. The most common is when people get so excited by a new frontier AI model that they shift focus, however subconsciously, from fighting for regulation of AI toward cheering on the company building the model and advocating for their success. Additionally, users of the most powerful new models have recently observed these models steering them away from the tasks they (the users) requested, in favor of tasks that the model itself finds more congenial.
Corollary: If you think modern LLMs are capable of scheming and superpersuasion, you definitely shouldn’t be using them.
But I think what you’re saying is that the collection of lines might not actually represent my subjective beliefs?
Something like that, yes.
in the actual interface I used, you could only see one line at a time, which was supposed to fix that issue. (The idea was to “sample many I.I.D. futures”, without thinking about collective behavior, if that makes sense.)
Ah, I see. I used the interface too, I just didn’t get the memo and kept looking at / thinking about the other lines I’d drawn.
I’m reminded of this 80,000 Hours article about making major life decisions.
One of my favourite studies ever is ‘Heads or Tails: The Impact of a Coin Toss on Major Life Decisions and Subsequent Happiness’ by economist Steven Levitt of ‘Freakonomics’.
Levitt collected tens of thousands of people who were deeply unsure whether to make a big change in their life. After offering some advice on how to make hard choices, those who remained truly undecided were given the chance to use a flip of a coin to settle the issue. 22,500 did so. Levitt then followed up two and six months later to ask people whether they had actually made the change, and how happy they were out of 10.
The causal effect of quitting a job is estimated to be a gain of 5.2 happiness points out of 10, and breaking up as a gain of 2.7 out of 10! This is the kind of welfare jump you might expect if you moved from one of the least happy countries in the world to one of the happiest, though presumably these effects would fade over time.
I suspect the effect is less about change being generally good and more about inertia being generally bad. It’s hard to commit to major life changes, even when we expect them to be positive, just because humans are risk-averse by nature. I’d also be willing to bet that many of those people would’ve quit their jobs or broken up with their partners sometime within the next year anyway — it’s just that the coin flip accelerated the process and forced them to stop dithering.
Having just played around with the model for the first time, I think this sort of scribbling is probably susceptible to cognitive biases like “I want the lines to evenly cover the space of strictly possible futures” or something. There are probably dozens of possible short-timelines futures, but once you scribble a single line to represent short timelines, it feels a little stupid to draw ten more lines covering the exact same timeline.
Also, I think your forecast was fine. Here’s some math.
Below is a basic exponential:
In which
If we take the METR graph at face value, time horizons double ~twice per year[2]:
And in April 2026, the state-of-the-art model was Mythos Preview, which could do ~3 hour tasks at 80% reliability:
So by October 2028, 2.5 years later, we would expect SOTA to have a time horizon of 96 hours:
And we don’t get >1 month time horizons until April 2030:
Alternatively, if we plug in METR’s exact values (
Reaching a >1 month time horizon by October 2028 would require timelines to double 3.2 times per year, aka every 3.75 months, aka almost twice as fast as reality. We could also be wrong about
If you want to play around with the math yourself, here’s a link on Desmos.
(Also, could you have factored acceleration into your scribbles? Probably. But then you wouldn’t be outside-view forecasting anymore, right?)
- ^
I’ll be using hours as my unit, but it doesn’t really matter what this unit is: you could use anything and the equation would still work.
- ^
I’m being generous here: their original doubling time claim was actually 7 months, but the math is cleaner this way.
- ^
Note that
- ^
Paul Graham makes a similar argument here: Putting Ideas Into Words.
I would guess that it just increases the radius of afflicted mosquitoes.
If you clear out all stagnant water, it seems possible that some of the mosquitoes within flying range of you will go find alternative water sources that are further away from you but still within flying range. But if you redirect some fraction of these mosquitoes to a bad water source, you’ve ensured that those mosquitoes effectively don’t breed at all. And OP suggests in another comment that this strategy is also useful to cull seasonal mosquito breeding, which seems to come in one distinct wave each year; if that’s the case, shrinking the size of the initial population growth should have a pretty big impact. The impact would also be proportionate to the degree to which mosquitos are capable of longer-distance migration; if they don’t move at all, the bucket seems pointless, but if they’re capable of moving more than a few miles, the bucket would also be helpful in catching stragglers from other regions.
(A cursory Google search seems to suggest that mosquito breeding cycles are year-round in the tropics but seasonal in more temperate climates. But also this information comes from a Reddit thread, so take it with a grain of salt.)
This is also part of why meditation and/or therapy are helpful for debugging. It’s hard to catch trapped priors in the moment, but if you learn to slow down your thought processes and/or bring up a situation later so a skilled interlocutor can analyze it, your brain will slowly release its vice grip on your experience and allow you to actually observe what’s going on (and decide whether you want it to continue.)
Hm. I’m tempted to come up with some reasons and defend my argument, but if I’m being honest, I think I just have a strong intuition that mass persuasion (specifically scheming/deception-based mass persuasion) is going to prove harder than robotics. People, culture, and societies just feel intensely chaotic and non-deterministic in a way that robotics isn’t. But I could be wrong!
The Hugging Face breach is probably not a clear warning shot. It might spur policymakers into action, but it seems like there are still a few mitigating factors preventing it from taking off—for example, some people who I respect aren’t taking it very seriously yet due to their strong distrust of OpenAI and Sam Altman. And since no one was directly harmed, we’re still left saying “What happens if capabilities increase further??” instead of “This is what happens if we don’t intervene right now,” which is obviously the stronger message.
In some ways, this is good: if we mobilize now, we’ll probably do so even more if we get an indisputably clear warning shot (e.g. an AI commits some act of terrorism). On the other hand, I do hope we’re not capitalizing on our goodwill early; there are some people out there who are determined to paint AI Safety people as perennial wolf-criers, and I worry that they’ll say the same about us this time.
I’d say right before. It seems to me that mass LLM adoption hasn’t significantly degraded the epistemic commons and in some ways is set to improve it; I suspect it’s partially due to:
The old Aristotle reason (that it’s easier to convince people of true things than it is to convince them of false things: see also Guided By The Beauty Of Our Weapons)
The fact that removing true facts from their pre-training datasets would make them worse
The fact that it’s hard to post-train them into believing specific false things without causing other kinds of emergent misalignment (e.g. MechaHitler Grok).
I am worried about large-scale persuasion (for more on this, see @dynomight’s post on the topic), but I’d have to see a lot more LLM adoption / far greater persuasion capabilities before I begin to get worried, and I think there are some reasonable ways to avoid it. Also, the longer we wait, the more we’ll develop anti-LLM persuasion cultural antibodies, though this may not be reassuring to those with short timelines.
Conversely, the robotics seems like it will be immediately and obviously disruptive as soon as it happens. And the robotics concern is roughly symmetric across timelines, because people with short timelines think ASI will invent super robots immediately and people with longer timelines expert the world to keep spinning long enough for better robots to be developed.
(Unrelated to its ranking, it’s also very a great starting point when talking to people, because “machine replaces man in rote physical task” has been the story of the past couple hundred years—no one denies that it’s a thing.)
Hmm I see. I think I agree that it would be fine in that particular case. I just think it’s dangerous to have a mindset like “I won’t be misled if I’m aware of the risks,” just because that mindset can justify staying in toxic environments, and that the norm should be avoided generally, even if it’s technically tolerable in low doses.
(Anecdotally, I’ve seen many people explicitly cite the signposting logic ⇒ spend too much time on X ⇒ fall victim to Twitter Brain ⇒ become epistemically (and emotionally) worse because of it.)