Anyone still want to build a nuke using AI, as per Yudowsky’s post earlier? OpenAI reckons nukes aren’t the answer, but Bioweapons are. https://openai.com/index/gpt-5-5-bio-bug-bounty/
daijin
Karma: 101
I feel like the defining characteristic of AI slop is that it invokes pauses and exclamation very often, and goes too shallow. I propose that shallow attention-seeking content is high reward and therefore self-reinforcing for human internet users → This becomes the majority of pre-AI internet content → AI learns to surface it as the statistically dominant form of writing.
mounting a campaign to improve the human sanity waterline could have disasterous race effects
hmm, what does this mean?
Great LessWrong posts often stick in people’s minds and continue to be reread and shared for years after they are published
I suspect that this is true not because Lesswrong is better than any other publishing platform, but rather because of a broader ‘rich get richer’ effect applied to good articles, and a survivorship bias.
Illustrative counterpoint: How many early-days-lesswrong articles were lost to time?Illustrative contrapoint: Rich Dad Poor Dad was not a lesswrong article and is still incredibly well known.
ergo I’d amend your ‘you should often aim for the final output of a project to be a LessWrong post’ to ‘to be a good lesswrong post’.Definitely share stuff on lesswrong though. :)
im not entirely sure ‘design skills’ and ‘technical skills’ are distinct, or really management skills.
let’s frame like this: given a problem:
- Physical skills would take the solution-as-a-concept and immanentize it as a solution-in-the-world
- Management skills would look at the universe of problems, and the universe of resources, and allocate accordingly
- Design skills will pore over the solution space and pick one that meets the problem parameters
- Technical skills… will pore over the solution space and pick one that meets the problem parameters
Ergo, design skills and technical skills are the same.
Perhaps you are missing the Problem-refiner skill: who will make very clear what the problem is, what are the things that need to remain as-they-are, what can be changed, and then feed it to the other skills. (One might argue that this is the Management skill; and I would agree—I’d just say the Management skill is allowed to invoke the other skills, whereas the problem-refiner skill is not.)
TL;DR.
What if the effect of AGI development would be our reform instead of our extinction?
There is a burden to prove not only that ‘some’ AGI development will be good for humanity (reforming, to use your words), but that all AGI cannot possibly lead to extinction. If someone creates a reforming-AI today, and then the next day, someone creates an evil AI, we will probably still all die.
1) hard to say. dont think yudowksy himself even could have told you trump was going to e.g. bomb iran by a specific date and time
2) use AI to create functional plans for a nuke. See this comment above https://www.lesswrong.com/posts/5CfBDiQNg9upfipWk/only-law-can-prevent-extinction?commentId=XgT7JRD3uZPNkiDnm
3) An honourable death requires people left standing to honor you. After evil ASI, there probably won’t be people left standing to honor you. Sorry :(
https://www.lesswrong.com/posts/n5TqCuizyJDfAPjkr/the-baby-eating-aliens-1-8
There’s an alien race in the above called the baby-eaters who eat their sentient offspring because this alien species is genetically unfortunate enough to produce thousands of immediately-sentient offspring at a time, and therefore they must cull their offspring.
More and more each day I think we are becoming like baby-eaters. Except of course we don’t murder our offspring by directly consuming them.
What is murder? What is consumption? It is the reclaiming of productive resource and the denial of future growth. When a poor person, having lived through years of their life giving what little they must to society in order to survive, dies on the street, there is another person that has been eaten by society.
We are eating each other, and often by the time someone has reached their preteens we know that they will be one of the ones that will be eaten. Therefore, in many ways, we are becoming baby-eaters.
However unlike the baby-eaters, not all of our society has normalized baby-eating to the point that we cannot see it as anything other than unquestionably good.
What would we ask of the baby-eaters? How should we act today?
Anyone remember a series of quick shorts posted by alexander scott (i think?) which includes a nation which includes prediction markets in its political process and a dictator takes over by opening a market that he will become the dictator in the next election cycle?
It’s been 15 years. Did you figure out how to be less scared?
The solution isn’t trying harder to be liked. It’s expanding your comfort with being disliked.
Social anxiety is an optimal response when there is a scarcity of other people to interact with. If you are meeting new 100 people every day, it doesn’t matter if 99 people dislike you, so long as you get another 100 new people tomorrow; because as long as you keep playing you will continue to gather people who like you.
If your total count of people to interact with is very small, then it suddenly becomes incredibly important to be not disliked, because you will quickly exhaust all your social prospects and be disliked by everyone.
I recently heard that thinking out loud is an important way for people to build trust (not just for LLMs) and this has helped me become more vocal. It has unfortunately not helped me become more correct, but I’m betting the tradeoff will be net positive in the long run.
go find people who are better than you by a lot. one way to quickly do this is to join some sort of physical exercise class e.g. running, climbing etc. there will be lots of people who are better than you. you will feel smaller.
or you could read research papers. or watch a movie with real life actors who are really good at acting.
you will then figure out, as @Algon has mentioned in the comments, that the narcissism is load-bearing, and have to deal with that. which is a lot more scary
game-theory-trust is built through expectation of reward from future cooperative scenarios. it is difficult to build this when you ‘dont actually know who or how many people you might be talking to’.
I did see the XKCD and I agree haha, I just thought your phrasing implied ‘optimize everything (indiscriminately)’.
When I say caching I mean retaining intermediate results and tools if the cost to do so is near free.
Nice. So something like grabbing a copy of swebench dataset, writing a pipeline that would solve those issues, then putting that on your CV?
I will say though that your value as an employee is not ‘producing software’ so much as solving business problems. How much conviction do you have that producing software marginally faster using AI will improve your value to your firm?
so you want to build a library containing all human writings + an AI librarian.
the ‘simulated planet earth’ is a bit extra and overkill. why not a plaintext chat interface e.g. what chatGPT is doing now?
of those people who use chatgpt over real life libraries (of course not everyone), why don’t they ‘just consult the source material’? my hypothesis is that the source material is dense and there is a cost to extracting the desired material from the source material. your AI librarian does not solve this.
I think what we have right now (“LLM assistants that are to-the-point” and “libraries containing source text”) serve distinct purposes and have distinct advantages and disadvantages.
LLM-assistants-that-are-to-the-point are great, but they
don’t exist-in-the-world, therefore sometimes hallucinate or provide false-seeming facts; for example a statement like “K-Theanine is a rare form of theanine, structurally similar to L-Theanine, and is primarily found in tea leaves (Camellia sinensis)” is statistically probable (I pulled it out of GPT4 just now) but factually incorrect, since K-theanine does not exist.
don’t exist in-the-world, leading to suboptimal retrieval. i.e. if you asked an AI assistant ‘how do I slice vegetables’ but your true question was ‘im hungry i want food’ the AI has no way of knowing that; and also the AI doesn’t immediately know what vegetables you are slicing, thereby limiting utility
libraries containing source text partially solve the hallucination problem because human source text authors typically don’t hallucinate. (except for every poorly written self-help book out there.)
from what I gather you are trying to solve the two problems above. great. but doubling down on ‘the purity of full text’ and wrapping some fake grass around it is not the solution.
here is my solution
atomize texts into conditional contextually-absolute statements and then run retrieveal on these statements. For example, “You should not eat cheese” becomes “eating excessive amounts of typically processed cheese over the long run may lead to excess sodium and fat intake”.
help AI assistants come into the world, while maintaining privacy
Another consequence of this is that inviting your friend to zendo is not weird, but inviting all your friends publically to zendo is.
‘Weirdness’ is not about being other from the group, it is about causing the ingroup pain, which happens to correlate to being distinct from the ingroup (weird). We should call them ingroup-pain-points.
Being loudly vegan is spending ingroup-pain-points, because being in front of someone’s face and criticising their behaviour causes them pain. Serving your friends tasty vegan food does not cause them pain and therefore incurs no ingroup-pain-points.
There is a third class of ingroup pain point that i will call ‘cultural pain point’. My working definition of ‘culture’ is ‘suboptimal behaviours that signal ingroup membership’. If you refuse to partake in suboptimal behavior, this does not cause you pain, but since you are now in a better position than others in the ingroup, you have now caused them pain. This is why you can be vilified for being vegan in certain ‘cultures’: you are being more optimal (healthier) relative to other people in a way that is (implicitly or explicitly) identified as a signalling-suboptimal-behaviour.
[winner of AI winner-takes-all]’s Basilisk?