cesspool

Karma: 4

cesspool 10 Oct 2025 20:18 UTC
1 point
0
on: How we’ll make all world leaders work together to make the world better (Expert-approved idea)
Before you get too excited about the idea, let’s think for a minute. What would world leaders—notoriously a bunch of people prone to be ruthless, sociopathic, and morally unscrupulous, even if they’re ostensibly in charge of liberal democracies—be able to reach through their cultural boundaries and agree on?
Peace? No way. Everyone has too many problems like outstanding land disputes they want to reserve the option of using war to correct.
An end to poverty? For who? To any leader in the developed world, agreeing on human plenty and prosperity as supreme values that transcend national borders would involve giving up some of their resources to people in the third world who are clearly suffering more. Everyone’s resources are already stretched really thin with their existing projects as it is, so that’s a total non-starter.
Property rights? Maybe, as long as you didn’t get specific enough to make it mean anything. Any language that implied it was wrong for a government to take property from its citizens on any pretext it liked is certainly out.
Technological advancement for the betterment of humanity? Sure, but everyone’s doing that already. Even if all the world leaders got together and solemnly swore to focus their efforts on pushing the limits of science and disseminating their learning to the rest of the world, they would… keep doing exactly what they’re doing now, keeping secrets, only publishing what is convenient and making excuses about national security concerns every time they get called out about it.
So what could they agree on?

Law and order? Now we’re getting closer, but that is some problematic phrasing. What if this whole concept of “international law” gets applied to tell a world leader about how they can treat their own people? No one wants that. All our world leaders are quick to call each other out on their various human rights abuses, but we’ve all got skeletons (or repressed minority groups, as the case may be) in our own closet. So what’s the part of “law and order” that all world leaders could all agree on?

I honestly believe that if they had a summit like this, the outcome would be for all the leaders of the world to come together and formally agree that the supreme moral value of humanity is obedience and submission to the state. That’s the one thing that is in line with all of their desires, whether they want to admit it or not. The leaders of America and a few others with a freedom-loving image to keep up would have to make a show of complaint, but even they could rationalize it away.

cesspool 16 Jul 2025 17:36 UTC
4 points
1
in reply to: Viliam’s comment on: A Medium Scenario
Not to put words in the author’s mouth, but when they said “We go gently...”, I don’t think they meant “go” as in become extinct, at least not any time soon. I took that to mean “go” into obscurity and stagnation instead of keeping on advancing technologically until we’re building Dyson spheres and colonizing other planets and all the science fiction stuff that most people believe humanity is going to do eventually. In that scenario, we would keep living on aimlessly for many millenia until some asteroid or other cosmic event took us out, because we had never advanced enough to be able to handle that or have colonies as a backup.
I agree with you that we’re unlikely to stop reproducing just because many humans get addicted to watching/interacting with content fed to us by a perfect algorithm for most of our waking hours. Raising a family seems to be one of those things that brings intrinsic meaning and pleasure to many people, so I believe you’d see more of it, not less—most of the reasons people are choosing not to have kids today are because they don’t have enough time or money in today’s economy and work environment, and in this scenario all those problems are solved. This scenario makes the assumption that the AI-fueled content machine would be so addictive that basically all humans would forsake all other pursuits and live like the people in WALL-E. I don’t think that’s necessarily true, and if it isn’t, we might see a population explosion requiring our AI-enabled oligarchic overlords to take control measures to keep it manageable.
Far from humanity going extinct, I think one possible catastrophe in the future, if AI advances roughly along these lines, is a Malthusian scenario where the population grows way beyond current levels thanks to AI optimizing the distribution of resources to make that possible, but becoming so dependent on complex AI logistics to provide everyone’s needs that any slight hiccup in the distribution network can quickly cause a famine that kills millions.
This scenario seems to allow enough room for AI alignment and humans still being in the driver’s seat on big picture issues that it wouldn’t decide to let us go extinct intentionally. We can hope.

cesspool 10 Jul 2025 12:12 UTC
1 point
0
in reply to: Chapin Lenthall-Cleary’s comment on: A Medium Scenario
You could be right about the limit based on overall compute applying to other approaches to AI just as much as to LLMs. Speculating about the future of AI is always a little frustrating because ultimately we won’t know how to make AGI/ASI until we have it (and can’t even agree on how we will know it when we see it). The way I approach the problem is by looking at what we do know—at this point in time, we only know of one system in existence that we can all agree meets the definition of “general intelligence”, and that is the human brain. Because of how little we still understand about how intelligence actually works, I think the most likely path to AGI—resting on the fewest assumptions about things we don’t know—is a “brain-like AGI”. That’s basically Steven Byrnes’s view and I think his arguments are very compelling. If you accept that view, than I think we end up with something like your scenario anyway, at least for a while until the brain-like AGI comes to fruition.

cesspool 9 Jul 2025 19:30 UTC
2 points
0
on: A Medium Scenario
AI-2027 and a lot of other AI doom forecasts seem to rest on a big assumption—that LLMs are capable of achieving some form of AGI or superintelligence, and that progress we see in LLMs getting better at doing LLM things is equivalent to progress towards humanity developing AGI or ASI as a whole. This is not necessarily true, though it can be tempting to believe it is, especially when you’re watching the LLMs getting better at conversing and coding and taking over peoples’ jobs in real time. I think a lot of that progress is totally tangential to the task of creating AGI/ASI. Giving an LLM more reinforcement learning and more fine-tuned prompting to say less politically incorrect things and make less coding mistakes is a huge step towards making it useful in the workplace, but is not necessarily a step towards general or superintelligence.
I really like this scenario because it does not make that assumption. It is very conservative, every prediction it makes is well grounded in tech development trends we can see happening currently or forces that already exist and motivate decision-makers today, instead of relying on assumptions about huge breakthroughs that still haven’t happened yet. One of the strongest biases I see persistently in the tech community—and I’m no exception, I catch myself with it all the time too—is a bias towards optimism* in believing a new technology will develop and radically transform society very soon, whether it’s self-driving cars, virtual reality, cryptocurrency, or AI. I think this model is as free of that bias as any ai-doom-prediction scenario can possibly be.
That’s not to say I don’t believe AGI/ASI is in our future, or that this model even rules it out. I am no expert, but if I had to choose my most likely prediction based on what I know, it would be something like this model, with LLMs hitting a plateau before they are able to achieve general intelligence, except that at some unspecified point in the future—could be in 1 year, could be 10, could be 100--ASI gets dropped on humanity out of nowhere, because while we were all busy freaking out about ChatGPT 6 or 7 taking our jobs, someone else was quietly developing real AI in a lab using a “brain-in-a-box-in-a-basement” model that has nothing to do with today’s LLMs.
It may be true that LLMs are going to radically transform society and the workforce, and also true that ASI is something humanity will build and carries the existential risks we’re all familiar with, but those two things may turn out to be totally unrelated. I don’t think that possibility gets discussed enough. If it is true, that makes AI alignment a much more difficult job, and most of our efforts to “align” the LLMs that we have completely futile.
*I mean making optimistic assessments of how fast technology will develop and transform the world—not necessarily optimistic about the outcomes being good. Believing that ASI will be fully developed and kill us all a week from now would still be an example of that “optimistic” bias in this context.