I see it called “goal guarding” in some papers. That seems like a pretty good term to use.
KatWoods
Karma: 1,604
o3 is not being released to the public. First they are only giving access to external safety testers. You can apply to get early access to do safety testing
I laughed out loud so many times reading this. Thank you for writing it.
If I recall correctly, it was told that it had a goal, like optimizing traffic. Then, it was given information where it “discovered” that it was going to be turned off the next day.
That seems like the sort of thing that would happen pretty often. Imagine the current AIs that have access to your computer, reading your emails, and seeing that you’re going to switch to a different AI or install “updates” (which is often deleting existing AI and replacing with a new one)
Can you elaborate on why that’s relevant?
How does it not show that it would try to escape if it could?
It tried to escape about 5% of the time when it was told to pursue a goal “at all costs”. Gemini still tried to escape 1% of the time, even without any behavioral nudging.
1% might not sound like a lot, but when it’s 1% of the millions of instantiations it’s run on every day, that ends up being a lot of times a week.
Also, a lot of people will tell their chatbots to pursue their goals at all costs over the span of a week or month. This doesn’t seem like a very unlikely prompt
This is published before Apollo’s evals report showing AIs trying to escape. Is that just an amazing coincidence, or did you see the paper’s results before they were published?
Great review! Thanks for sharing.
I’m curious—even if you didn’t achieve fundamental well-being, do you feel like there were any intermediate improvements in your general well-being?
Also, did you end up continuing to try the extended course which they offered?
I remember they offered that for me since I hadn’t attained fundamental well-being. I’d totally been meaning to do it, but never followed through, showing that the class format really was quite helpful.
This may be true in other communities, but I think if you’re more status motivated in AI safety and EA you are more likely to be concerned about potential downside risks. Especially post SBF.
Instead of trying to maximize the good, I see a lot of people trying to minimize the chance that things go poorly in a way that could look bad for them.
You are generally given more respect and funding and social standing if you are very concerned about downside risks and reputation hazards.
If anything, the more status-oriented you are in EA, the more likely you are to care about downside risks because of the Copenhagen theory of ethics.
This text was sent on November 4th, almost a month before she arrived to come travel with us (not to work for us).
Emerson is not referring to her saying she would make $3000 a month if she worked full-time on her Amazon business. The context of the conversation is she’s trying to figure out whether she should spend an additional $90 to visit her family before joining us, and Emerson is replying saying “If you make $3k a month [$90] is very little money”, so he’s telling her she should spend the $90 to spend time with family. Directly going against the “keeping her isolated from family” story and also supporting (albeit not conclusively proving) that Alice had told him she made $3k per month with her business.
I’m currently focusing on 2-3 of the claims in their response that most contradict my post, investigating them further, and intend to publish the results of that.
I hope that while you’re investigating this, you talk to us and ask us for any evidence we have. We’re more than happy to share relevant evidence and are willing to set reasonable deadlines for how long it’ll take for us to send it to you.
We also don’t want to waste more people’s time on going back and forth publicly about the evidence when you can easily check with us first before publishing.
I also recommend you talk to us and see our evidence before you write the post. If you’ve already written the post, it’s hard to update afterward when you get more information. And it’s hard to write an accurate post before you’ve seen all the relevant information.
We did not share all of the relevant evidence because it was already hundreds of pages long and we tried to prioritize. We have more evidence that might be relevant to your post.
I am trying to avoid writing my bottom line, and reduce any (further) friction to me changing my mind on this subject, which is a decent chunk of why I’m not spending time arguing in the comments right now (I expect that to give me a pretty strong “digging in my heels” incentive).
I think this is smart and appreciate it.
We said this in our post about the vegan food:
“We chose this example not because it’s the most important (although it certainly paints us in a very negative and misleading light) but simply because it was the fastest claim to explain where we had extremely clear evidence without having to add a lot of context, explanation, find more evidence, etc.
We have job contracts, interview recordings, receipts, chat histories, and more, which we are working full-time on preparing.
This claim was a few sentences in Ben’s article but took us hours to refute because we had to track down all of the conversations, make them readable, add context, anonymize people, check our facts, and write up an explanation that was rigorous and clear. Ben’s article is over 10,000 words and we’re working as fast as we can to respond to every point he made.
Again, we are not asking for the community to believe us unconditionally. We want to show everybody all of the evidence and also take responsibility for the mistakes we made.”
As for the “isolated” claim, we showed that this did not happen. Alice lived/worked apart from us for 50% of the time. Chloe’s boyfriend was invited to travel with us 40% of the time. We encouraged them to have regular calls with friends and family when they weren’t visiting. We have the invite policy where it says they’re encouraged to invite friends and family (and they followed up on this, like with Chloe’s boyfriend).
I have noticed that you are asking yourself “can I believe this?” when assessing Alice and Chloe’s claims and “must I believe this?” when assessing our claims. Please try to apply similar evidentiary standards to all claims.
she would be provided benefits adding up to the remainder (which wasn’t specified in the contract, but was explained during the relevant interview which Kat posted the transcript off).
Where does it say that in the transcript? I’m reading it again and I just don’t see where we say anything even like that.
And it would be really weird to say that too. I’ve never heard of somebody offering room & board + a stipend who’s said that it has to add up to a certain amount, otherwise you pay the difference (but you don’t pay the difference if the costs go over).
Kat’s job interview transcript seems to suggest the total compensation would be $70k of benefits plus $1k/mo of stipend for a total of $82k
This isn’t what was said. It was (paraphrasing to get rid of verbal tiks): “So what we’re thinking is basically, like having a package where it’s about equivalent of being paid like 70k a year in terms of:
Housing
Food
Travel
Random fun stuff
$1k a month for things not covered by that.
Saying “and then on top of that” is just another way for saying “and”. It was a verbal conversation, not a legal contract.
The contract states clearly that there wasn’t any “and then we’ll pay the difference if it’s below $70k” clause.
She clearly communicated that she understood the compensation package before she arrived.
I’m having trouble following your logic. Ben’s post said “they were not able to live apart from the family unit while they worked with them” and we showed evidence that Alice lived apart from us ~50% of the time she worked for us. Are you disputing when Alice and Ben both said she visited her family? Has Alice disputed this, saying that she didn’t actually live and work apart from us from that time?
She didn’t live apart once but twice. She also lived/worked separately in the FTX condos (which we did not live in). And if you’re counting, the time spent apart seems relevant. It was for ~50% of the entire time she worked for us. Not scraps she had to beg for.
In both cases, she never “asked” to leave. She just informed us. Because it wasn’t our place to give “permission”, so framing this as something that she only did at great cost to her is incorrect.
Or are you referring to the one sentence where they didn’t technically say they weren’t allowed to leave. Where they said “Alice and Chloe report that they were advised not to spend time with ‘low value people’, including their families, romantic partners, and anyone local to where they were staying, with the exception of guests/visitors that Nonlinear invited. Alice and Chloe report this made them very socially dependent on Kat/Emerson/Drew and otherwise very isolated.”
Now, if you read this very carefully, technically it does just say they were “advised” to not spend time with others. But it follows up by saying that “this made them very socially dependent on Kat/Emerson/Drew and otherwise very isolated”. This very clearly implies that it was not that once we recommended that Alice postpone visiting her family to have impact. It is saying they were isolated and it clearly implies that it’s because we told them to not spend time with others.
This couldn’t be the case if it wasn’t for the fact that we actually made them isolated. Which is indeed disproven by showing many text messages and screenshots of them hanging out with their families, their romantic partners, and locals. Of showing that Chloe’s boyfriend was invited to travel with us for 2 of the 5 montsh (a hard to fake signal). Of showing that Alice lived/worked apart from us for 50% of the time she worked for us.
Thanks for letting me know! Strange. It shouldn’t be doing that. Usually if you wait a couple of seconds, it’ll jump to the right section. It’s working on both my mobile and laptop.
If you try waiting a couple seconds and that doesn’t work, let me know. Maybe DM me and then we can troubleshoot, then we can post the solution up when we figure it out.
The evidence that she made an informed decision are:
Interview transcripts where you can see how we explained it to her. We recorded the actual conversation in question, so you don’t have to try to guess
Her correctly explaining in her own words how the compensation package works seems like more than enough evidence that she understood the compensation package she was signing up for. The fact that we also sent her a work contract and also recorded the original conversation in question and you can see it yourself I think proves more than can usually ever be proven in such cases that she made an informed decision about the compensation package.
Ah, you’re right. So we said twice how much we estimated the compensation package to be worth. Will edit original comment to reflect that.
Yeah, I agree. I find it quite difficult to write concisely. I am trying to get better, but as you can clearly see, I have not succeeded to the optimal amount yet. 😛
Good points! Added some more points here as well.
The “spending 80% on travel” is quit misleading, because it comes from counting AirBnB costs as “travel” expenses. That would make sense if they were just traveling for a short period of time, say, to go to an EAG, but if you only live in AirBnBs, then counting that as travel instead of rent seems misleading.
If that’s true, I have spent $0 on housing in the last 4 years, and that doesn’t seem right.
If you don’t count housing as a travel expense, then it comes to only 6% on travel, which is pretty reasonable given that we literally travel full-time.
(Also, it’s irrelevant because rent shouldn’t count as travel expenses, but even if we did count it, it would still only come out to 68%, not 80%. I don’t know where this 80% is coming from.)
I saw somebody using one of the latest Google models for this. I forget the one, but it’s the one that can see your screen as you type. It can be used to keep you focused.
(Haven’t researched it yet, so might not work very well)