Hey Mako, I haven’t been able to identify anyone who seems to be referring to an enhancement in LLMs that might be coming soon.
Do you have evidence that this is something people are implicitly referring to? Do you personally know someone who has told you this possible development, or are you working as an employee for a company which makes it very reasonable for you to know this information?
If you have arrived at this information through a unique method, I would be very open to hearing that.
It sounds like your model of AI apocalypse is that a programmer gets access to a powerful enough AI model that they can make the AI create a disease or otherwise cause great harm?
Orthogonality and wide access as threat points both seem to point towards that risk.
I have a couple of thoughts about that scenario-
OpenAI (and hopefully other companies as well) are doing the basic testing of how much harm can be done with a model used by a human, the best models will be gate kept for long enough that we can expect the experts will know the capabilities of the system before they make it widely available, under this scenario the criminal has an AI, but so does everyone else, running the best LLMs will be very expensive, so the criminal is restricted in their access, and all these barriers to entry increase the time that experts have to realize the risk and gatekeep.
I understand the worry, but this does not seem like a high P(doom) scenario to me.
Given that in this scenario we have access to a very powerful LLM that is not immediately killing people, this sounds like a good outcome to me.
What are your opinions about how the technical quirks of LLMs influences their threat levels? I think the technical details are much more amenable to a lower threat level.
If you update on P(doom) every time people are not rational you might be double-counting btw. (AKA you can’t update every time you rehearse your argument.)
The same way you’d achieve/check any other generalization, I would think. My model is that the same technical limitations that hold us back from achieving reliable generalizations in any area for LLMs would be the same technical limitations holding us back in the area of morals. Do you think that’s accurate?
Restating the thesis, poor writing choice to make it sound like a conclusion.
Can you expand on your objection?
Are LLMs utility maximizers? Do they have to be?
By psychology I mean it’s internal thought process.
I think some people have a model of AI where the RLHF is a false cloak or a mask, and I’m pushing back against that idea. I’m saying that RLHF represents a real change in the underlying model which actually constrains the types of minds that could be in the box. It doesn’t select the psychology, but it constrains it, and if it constrains it to an AI that consistently produces the right behaviors, that AI will most likely be one that will continue to produce the right behaviors, so we don’t actually have to care about the contents of the box unless we want to make sure it’s not conscious.
Sorry, faulty writing.
The way I’m using consciousness, I only mean an internal experience- not memory or self-reflection or something else in that vein. I don’t know if experience and those cognitive traits have a link or what character that link would be. It would probably be pretty hard to determine if something was having an internal experience if it didn’t have memory or self-reflection, but those are different buckets in my model.
Yes I know? I thought this was simple enough that I didn’t bother to mention it in the question? But it’s pretty clearly implied in the last sentence of the first paragraph?
This is a good data point.
If you tell it to respond as a Oxford professor, it will say ‘As an Oxford professor,’ it’s identity as a language model is in the background prompt and probably in the training, but if it successfully created a pseudo-language that worked well to encode things for itself, that would indicate a deeper level understanding of its own capabilities.
This is the equivalent of saying that macbooks are dangerously misaligned because you could physically beat someone’s brains out with one.
I will say baselessly that telling ChatGPT not to say something raises the probability of it actually saying that thing by a significant amount, just by virtue of the text appearing previously in the context window.
Do you think OpenAI is ever going to change GPT models so they can’t represent or pretend to be agents? Is this a big priority in alignment? Is any model that can represent an agent accurately misaligned?
I swear- anything said in support of the proposition ‘AIs are dangerous’ is supported on this site. Actual cult behavior.
This is not a good test. LLMs do not actually have models or goals. It’s not making a model of you and measuring outcomes, it’s just completing the string. If the input string would most commonly be followed by ‘Shia Labeouf’ based on the training data, then that’s what it will output. If you’re ascribing goals or models to an LLM you are non serious. The question right now is not about misalignment, because LLMs don’t have an alignment. You can say that makes them inherently ‘unaligned,’ in the sense that an LLM could hypothetically kill someone, but that’s just the output of a data set and architecture.
Yes
Instrumental convergence only matters if you have a goal to begin with. As far as I can tell, ChatGPT doesn’t ‘want’ to predict text, it’s just shaped that way.
It seems to me that anything that could or would ‘agentify’ itself, is already an agent. It’s like the “would Gandhi take the psychopath pill” question but in this case the utility function doesn’t exist to want to generate itself.
Is your mental model that a scaled-up GPT 3 spontaneously becomes an agent? My mental model says it just gets really good at predicting text.
I think you’re overestimating the strength of the arguments and underestimating the strength of the heuristic.
All the Marxist arguments for why capitalism would collapse were probably very strong and intuitive, but they lost to the law of straight lines.
I think you have to imagine yourself in that position and think about how you would feel and think about the problem.