What would you mean by ‘stays at human level?’ I assume this isn’t going to be any kind of self-modifying?
FinalFormal2
Karma: 59
What does it mean for an AI to ‘become self aware?’ What does that actually look like?
Is there reason to believe 1000 Einsteins in a box is possible?
You need to think about your real options and expected value of behavior. If we’re in a world where technology allows for a fast takeoff world and alignment is hard, (EY World) I imagine the odds of survival with company acceleration is 0% and the odds of survival without is 1%.
But if we live in a world where compute/capital/other overhangs are a significant influence in AI capabilities and alignment is just tricky, company acceleration would seem like it could improve the chances of survival pretty significantly, maybe from 5% to 50%.
These obviously aren’t the only two possible worlds, but if they were and both seemed equally likely, I would strongly prefer a policy of company acceleration because the EV for me breaks down way better over the probabilities.
I guess ‘company acceleration’ doesn’t convey as much information or sell as well which is why people don’t use that phrase, but that’s the policy they’re advocating for- not ‘hoping really hard that we’re in a slow takeoff world.’
That seems like a useful heuristic-
I also think there’s an important distinction between using links in a debate frame and in a sharing frame.
I wouldn’t be bothered at all by a comment using acronyms and links, no matter how insular, if the context was just ‘hey this reminds me of HDFT and POUDA,’ a beginner can jump off of that and get down a rabbit hole of interesting concepts.
But if you’re in a debate frame, you’re introducing unnecessary barriers to discussion which feel unfair and disqualifying. At its worst it would be like saying: ‘youre not qualified to debate until you read these five articles.’
In a debate frame I don’t think you should use any unnecessary links or acronyms at all. If you’re linking a whole article it should be because it’s necessary for them to read and understand the whole article for the discussion to continue and it cannot be summarized.
I think I have this principle because in my mind you cannot not debate so therefore you have to read all the links and content included, meaning that links in a sharing context are optional but in a debate context they’re required.
I think on a second read your comment might have been more in the ‘sharing’ frame than I originally thought, but to the extent you were presenting arguments I think you should maximize legibility, to the point of only including links if you make clear contextually or explicitly to what degree the link is optional or just for reference.
This is a fantastic project! Focus on providing value and marketing, and I really think this could be something big.
LessWrong continues to be nonserious. Is there some sort of policy against banning schizophrenic people in case that encourages them somehow?
AND conducted research on various topics
Wow that’s impressive.
lol
I don’t like the number of links that you put into your first paragraph. The point of developing a vocabulary for a field is to make communication more efficient so that the field can advance. Do you need an acronym and associated article for ‘pretty obviously unintended/destructive actions,’ or in practice is that just insularizing the discussion?
I hear people complaining about how AI safety only has ~300 people working about it, and how nobody is developing object level understandings and everyone’s thinking from authority, but the more sentences you write like: “Because HFDT will ensure that it’ll robustly avoid POUDA?” the more true that becomes.
I feel very strongly about this.
To restate what other people have said- the uncertainty is with the assumptions, not the nature of the world that would result if the assumptions were true.
To analogize- it’s like we’re imagining a massive complex bomb could exist in the future made out of a hypothesized highly reactive chemical.
The uncertainty that influences p(DOOM) isn’t ‘maybe the bomb will actually be very easy to defuse,’ or ‘maybe nobody will touch the bomb and we can just leave it there,’ it’s ‘maybe the chemical isn’t manufacturable,’ ‘maybe the chemical couldn’t be stored in the first place,’ or ‘maybe the chemical just wouldn’t be reactive at all.’
[Question] When does an AI become intelligent enough to become self-aware and power-seeking?
[Question] What are the arguments for/against FOOM?
[Question] What’s the consensus on porn?
[Question] What are some subjects with unexpectedly high utility?
I think you’re overestimating the strength of the arguments and underestimating the strength of the heuristic.
All the Marxist arguments for why capitalism would collapse were probably very strong and intuitive, but they lost to the law of straight lines.
I think you have to imagine yourself in that position and think about how you would feel and think about the problem.
Hey Mako, I haven’t been able to identify anyone who seems to be referring to an enhancement in LLMs that might be coming soon.
Do you have evidence that this is something people are implicitly referring to? Do you personally know someone who has told you this possible development, or are you working as an employee for a company which makes it very reasonable for you to know this information?
If you have arrived at this information through a unique method, I would be very open to hearing that.
It sounds like your model of AI apocalypse is that a programmer gets access to a powerful enough AI model that they can make the AI create a disease or otherwise cause great harm?
Orthogonality and wide access as threat points both seem to point towards that risk.
I have a couple of thoughts about that scenario-
OpenAI (and hopefully other companies as well) are doing the basic testing of how much harm can be done with a model used by a human, the best models will be gate kept for long enough that we can expect the experts will know the capabilities of the system before they make it widely available, under this scenario the criminal has an AI, but so does everyone else, running the best LLMs will be very expensive, so the criminal is restricted in their access, and all these barriers to entry increase the time that experts have to realize the risk and gatekeep.
I understand the worry, but this does not seem like a high P(doom) scenario to me.
Given that in this scenario we have access to a very powerful LLM that is not immediately killing people, this sounds like a good outcome to me.
What are your opinions about how the technical quirks of LLMs influences their threat levels? I think the technical details are much more amenable to a lower threat level.
If you update on P(doom) every time people are not rational you might be double-counting btw. (AKA you can’t update every time you rehearse your argument.)
I really relate to your description of inattentive ADHD and the associated degradation of life. Have you found anything to help with that?