My thought on it: culture and adoption lags capabilities, A LOT. By far most people don’t know there’s models more capable than free-tier Chat Jippity and have never engaged with something Astra/Fable class. To understand the frontier is one massive hop, and to figure out what to do with such capabilities is yet another massive hop. You can’t forget the people active in the AI communities are fractions of a percent, the genpop largely doesn’t comprehend the state of the art. It’s not a strike on anyone’s character, it just is. It’ll take time.
marquis_de_sod
Karma: 80
x-risk discussion becoming partisan is in and of itself a contributor to x-risk. ironic
Trump on a slowdown, as of Monday morning: (TLDR: No)
https://truthsocial.com/@realDonaldTrump/posts/117269745153543631
Tangential to your post, but the “doomsday clock” was at seven minutes to midnight during the Cuban missile crisis, and today sits at 85 seconds. I find it hard to believe you could find a reasonable person who would say we are at a higher risk of nuclear annihilation than during the peak of the Cold War. In your own post you reference the Petrov incident, during which nuclear-armed missiles were very nearly fired and stopped by the actions of one man, and at that time it was still at three minutes.
The continued existence is of the doomsday clock is, to me, a very explicit case of confusing the map for the territory and letting clock-as-a-political-tool take over clock-as-a-measurement.
This is glossing over the fact that if you can’t fit the weights into storage you’re not serving the model, ever. It’s an all or nothing thing, trillions of parameters is going to be in the terabyte+ range so you’re looking at late eighties/early nineties (optimistically) level of tech to conceivably serve a single model, extremely slowly, off the footprint of an entire data center—ONE instance. At best you get to speed through what, forty years? Otherwise you’re replaying the first 960 1:1.
What do you define as “escaping heat death”? A chunk of reversible computronium looping through the same routine for eternity doesn’t seem like an elevated state above the stagnation of heat death.
>but the bursty engine has a higher peak power than the continuous engine, meaning it is heavier and harder to build
Yes-but… that’s an engineering problem, not a fundamental physics/”tyranny of the rocket”-type problem as presented in the OP
No mirror is 100% reflective, some light is always absorbed.
Doesn’t this assume constant, unrelenting acceleration? If the radiators can only dump 1/n the heat of the propulsion system, wouldn’t it be possible to burn for one second and coast for n-1 seconds to bleed off the rest, repeating ad infinitum? The only constraint to max speed in both situations is drag from the interstellar medium, and with the higher effective constant acceleration you still run into that issue, albeit at a higher top speed.
This seems like it may create pressure towards covert development and “cheating” the release process.
transhumanism vs posthumanism basically
You could say that about any time period in history.
marquis_de_sod’s Shortform
New post from Asterisk Mag: “Beware the Permanent Periphery”
TL;DR: The idea of the “permanent underclass” applied to the middle powers of the world—frontier model training not only accelerates its own progress, but also tends to eat up all the supply-inflexible resources it relies on, (high power GPUs, HBM, etc) meaning that countries outside of the US and China may be (effectively) permanently locked out of the frontier.
I’ve been doing some research on this on my own, sending my agents to pull foreign-language sources from Japan, Korea, EU countries—the mood over there, if the translations are to be believed, seems to mirror these predictions. People are anxious. Reuters has quoted Japanese officials as fearing ending up as an “AI colony”. There’s real impetus to close the gap, but short of some unprecedented alliance of all/most of the middle powers into some international AI lab, I’m not sure how any one country will make it to the frontier.
I expect holding back high-performance models for internal use to become a default behavior now that large-scale distillation is running rampant.
My intuition is that what’s holding us back from it is measurement, not analysis—ie if we could extract a sufficiently high quality signal of brain state/activity, the rest of the problem will fall quickly. The state of the art currently seems to be either noninvasive highly sensitive magnetic measurements, or Neuralink-style “stick a couple wires in them”, neither of which seem to be yielding much, at least by way of loud/mainstream advancements. Imagine giving a modern CPU to a scientist from the 1930s—theoretically they have the ability to understand it, if you gave them a diagram and explained “so and so is an AND gate, &c”—but just by looking at it, running voltage through it, even imaging it via x-rays wouldn’t be enough—even though the science of that era is entirely able to explain and model the behavior thereof on an abstract level, you’d never be able to because you can’t get a good look at the damn thing to be able to explain it in the first place.
Average person not inculcated with a steady diet of sci-fi media—and I’m not talking about butter-soft Star Wars stuff, but actual hard sci-fi in the vein of your Egans and Baxters—tends to lack the ability to imagine/perceive such possibilities as RSI and therefore defaults to “nothing ever happens” as a response, however ill-informed that reaction may be.
>plural identity
On a tangent, is there any empirical evidence that such a thing actually exists? A cursory web search led me to believe not.
>Do you extend this argument to a human piercing the veil of privacy around their own mind?
Assuming no duress and a clear and knowing mind, I would see “user is doing it to themself” and overwhelmingly assume self-given consent for introspection… (side note does consent even exist as a concept when the actor and patient are the same person??) the line is when others do it to you. Psychotherapy also doesn’t generally work without the active participation of the client.
Too wordy. Needs a pithy tagline/hook not an explainer. Anyone looks at that and their eye will slide right off that wall of text.