Another hypothesis: geopolitical actors have limited bandwidth and their even considering your argument of stopping AI progress gives them less bandwidth to deal with <all the other sht they’re fighting against> and so purely by virtue of this, they don’t want to all come together and solve a problem that you think is important.
daijin
Karma: 97
My argument is that you are using the notion of superintelligence to abstract away too many things. What is a superintelligent LLM? But I think everyone else in the comments section is already grilling you on this.
Really easy fix there. We can just, by definition, define a superintelligent AI as one that does not perform action X anyway.
But I noticed that you mentioned super-LLM instead of superintelligence. Are we saying a superintelligence is one that doesn’t accidentally delete humanity, because it has an in-the-world notion of what that means?
我们停AI的话，他们会吗？
[winner of AI winner-takes-all]’s Basilisk?
Anyone still want to build a nuke using AI, as per Yudowsky’s post earlier? OpenAI reckons nukes aren’t the answer, but Bioweapons are. https://openai.com/index/gpt-5-5-bio-bug-bounty/
I feel like the defining characteristic of AI slop is that it invokes pauses and exclamation very often, and goes too shallow. I propose that shallow attention-seeking content is high reward and therefore self-reinforcing for human internet users → This becomes the majority of pre-AI internet content → AI learns to surface it as the statistically dominant form of writing.
mounting a campaign to improve the human sanity waterline could have disasterous race effects
hmm, what does this mean?
Great LessWrong posts often stick in people’s minds and continue to be reread and shared for years after they are published
I suspect that this is true not because Lesswrong is better than any other publishing platform, but rather because of a broader ‘rich get richer’ effect applied to good articles, and a survivorship bias.
Illustrative counterpoint: How many early-days-lesswrong articles were lost to time?Illustrative contrapoint: Rich Dad Poor Dad was not a lesswrong article and is still incredibly well known.
ergo I’d amend your ‘you should often aim for the final output of a project to be a LessWrong post’ to ‘to be a good lesswrong post’.Definitely share stuff on lesswrong though. :)
im not entirely sure ‘design skills’ and ‘technical skills’ are distinct, or really management skills.
let’s frame like this: given a problem:
- Physical skills would take the solution-as-a-concept and immanentize it as a solution-in-the-world
- Management skills would look at the universe of problems, and the universe of resources, and allocate accordingly
- Design skills will pore over the solution space and pick one that meets the problem parameters
- Technical skills… will pore over the solution space and pick one that meets the problem parameters
Ergo, design skills and technical skills are the same.
Perhaps you are missing the Problem-refiner skill: who will make very clear what the problem is, what are the things that need to remain as-they-are, what can be changed, and then feed it to the other skills. (One might argue that this is the Management skill; and I would agree—I’d just say the Management skill is allowed to invoke the other skills, whereas the problem-refiner skill is not.)
TL;DR.
What if the effect of AGI development would be our reform instead of our extinction?
There is a burden to prove not only that ‘some’ AGI development will be good for humanity (reforming, to use your words), but that all AGI cannot possibly lead to extinction. If someone creates a reforming-AI today, and then the next day, someone creates an evil AI, we will probably still all die.
1) hard to say. dont think yudowksy himself even could have told you trump was going to e.g. bomb iran by a specific date and time
2) use AI to create functional plans for a nuke. See this comment above https://www.lesswrong.com/posts/5CfBDiQNg9upfipWk/only-law-can-prevent-extinction?commentId=XgT7JRD3uZPNkiDnm
3) An honourable death requires people left standing to honor you. After evil ASI, there probably won’t be people left standing to honor you. Sorry :(
https://www.lesswrong.com/posts/n5TqCuizyJDfAPjkr/the-baby-eating-aliens-1-8
There’s an alien race in the above called the baby-eaters who eat their sentient offspring because this alien species is genetically unfortunate enough to produce thousands of immediately-sentient offspring at a time, and therefore they must cull their offspring.
More and more each day I think we are becoming like baby-eaters. Except of course we don’t murder our offspring by directly consuming them.
What is murder? What is consumption? It is the reclaiming of productive resource and the denial of future growth. When a poor person, having lived through years of their life giving what little they must to society in order to survive, dies on the street, there is another person that has been eaten by society.
We are eating each other, and often by the time someone has reached their preteens we know that they will be one of the ones that will be eaten. Therefore, in many ways, we are becoming baby-eaters.
However unlike the baby-eaters, not all of our society has normalized baby-eating to the point that we cannot see it as anything other than unquestionably good.
What would we ask of the baby-eaters? How should we act today?
Anyone remember a series of quick shorts posted by alexander scott (i think?) which includes a nation which includes prediction markets in its political process and a dictator takes over by opening a market that he will become the dictator in the next election cycle?
It’s been 15 years. Did you figure out how to be less scared?
The solution isn’t trying harder to be liked. It’s expanding your comfort with being disliked.
Social anxiety is an optimal response when there is a scarcity of other people to interact with. If you are meeting new 100 people every day, it doesn’t matter if 99 people dislike you, so long as you get another 100 new people tomorrow; because as long as you keep playing you will continue to gather people who like you.
If your total count of people to interact with is very small, then it suddenly becomes incredibly important to be not disliked, because you will quickly exhaust all your social prospects and be disliked by everyone.
I recently heard that thinking out loud is an important way for people to build trust (not just for LLMs) and this has helped me become more vocal. It has unfortunately not helped me become more correct, but I’m betting the tradeoff will be net positive in the long run.
go find people who are better than you by a lot. one way to quickly do this is to join some sort of physical exercise class e.g. running, climbing etc. there will be lots of people who are better than you. you will feel smaller.
or you could read research papers. or watch a movie with real life actors who are really good at acting.
you will then figure out, as @Algon has mentioned in the comments, that the narcissism is load-bearing, and have to deal with that. which is a lot more scary
game-theory-trust is built through expectation of reward from future cooperative scenarios. it is difficult to build this when you ‘dont actually know who or how many people you might be talking to’.
Should cavemen have worked on reducing their carbon footprint?