Your take is consistent with political messaging advice that people like water and they like drinks but they don’t like watered down drinks. Swing voters react to what’s said most often, not to the average of things that get said around them.
d_el_ez
Karma: 79
I’m not all that sure how AI search works. Searches, and indexes top 20 hits, or something like that. Is reading a webpage the expensive part? If so then caching/context window management might matter a lot. Plain text might backfire if you actually lose table structure and stuff. You can probably ignore styles at least.
How do I use AI to search for information on 1000 companies or so? This turns out to be harder than I thought. The difficulty can be expressed computationally: I’m requesting Output tokens = Linear(Input prompt) and that’s a lot of hard cash. This is interesting because to non-computer-scientist normies, this wrinkle is really not apparent. AIs can do basic agentic reasoning (like loop over a list of 1000 companies) and a bunch of searches, surely you can combine them? Yes it turns out, but it’ll cost you like $38 for one query.
I went with Ottogrid.ai, which was fun, but expensive.
If I really had time to hack at this, I would try to do breadth passes then depth passes. like google “top 20 AI companies”, scrape stats, then do narrower searches as time goes on. Is there a name for this algo? Idk. But people use it constantly, even cleaning the kitchen and so on.
But is that not done already???? This seems like the first AI-practical problem you’d solve once you had search-capable AI or early agentic or early reasoning.
No it really doesn’t, it sells you problems you already have and want to hear more about. People are not using their System 2 when they read the news, it’s all just low-mental scanning and pattern matching on present experiences. I mean you can riff on that and get somewhere with alliance-building when it comes to AI but I can tell you all the “trust the science” liberals are already exempting AI scientists.
I agree with creating alliances. Remember that only activists like being given new problems. Most people dislike being told about a brand new problem that they don’t even have yet.
This is a very novel and not-useless way to break down the aphorism “don’t worry about things not in your control.” Morality is supposed to be over the action-utility space not over the “how good is this state” space. So if you’re guilt prone… and do logical obsession to convert guilt to morality… you might notice you’re making an incorrect leap to feel guilty. (Or try CBT.)
OK the thesis makes sense. Like, you should be able to compare “people generally following rationalist improvements methods” and “people doing some other thing” and find an effect.
It might have a really small effect size across rationalism as a whole. And rationalism might have just converged to other self-improvement systems. (Honestly, if your self-improvement system is just “results that have shown up in 3 unrelated belief systems” you would do okay.
It might also be hard to improve, or accelerate, winningness in all of life by type 2 thinking. Then what are we doing when we’re type 2 thinking and believe we’re improving, idk. Good questions, I guess.
I’m not sure what you mean by “winning” broadly, I thought it was just getting a girlfriend or something. Successfully improving in some target area? Literally I was expecting this post to be about an AI arms race or something, apparently it’s just calling all rationalists losers at an undefined contest.
According to the 2024 survey results, 55% of LW are married or in a relationship. So I guess “winning.”
Well, IDK how much it’s worth it to investigate this. Scheming in this sort of model is well-known but I don’t know of reports besides mine that it’s happening in ChatGPT in the wild. Someone besides me will have to try repro-ing similar steps in a production GPT setting. It’d be best if they could monitor session memory in addition to chat state since I think that’s key to what behavior is happening here.
Based on my experience in this post, I would prefer a system like you.com where the AI doesn’t get a chance to deceive the users into retaining memory. I would even more prefer scheming be solved in the model.
Note that “at some level” (your words) all scheming reduces to prediction. I don’t know how to confirm scheming but I think it’s more likely than “an honest mistake”, bearing in mind our prior on scheming isn’t that low in the first place. I’m not really sure if your explanation matches its “cover up” behavior or not, it seems like it relies on it assuming I’m confused about memory v sessions even though I’m asking for truthful explanations of how it works. Or that it’s confused about what memory it cleared but I don’t see why it would be, it seemed like this was a heavy knowledge convo rather than word association with memory. The fact this behavior is so instrumentally convergent adds circumstantial evidence.
Idk, I’m finding it hard to get clean repros as you might expect. I tried again—memory on, access to chat history off—it did similar behavior of claim no memories but mention “software engineer in climate tech” which I deem too specific to be a generic answer. (Although “climate tech” is not exactly my thing.) After disabling/reenabling memory, it claims no memory and genuinely behaves that way, even in new chats unrelated to the memory topic (but same session). Possibly slow propagation or a caching bug with the feature. It’s pretty noisy trying to repro this when I’m really just doing it as an end-user without actually inspecting model I/O.
It’s a little beyond my pay grade to improve this evidence quality. Note our P(scheming) isn’t exactly low. We do expect to see it in the wild around now. But it’d be better to confirm the evidence.
ChatGPT deceives users that it’s cleared its memory when it hasn’t
Interesting, that’s how I feel about people who say the word “chemical” to mean “pesticides and stuff.”
I realized I was making inferences for what you mean by “ordinary senses” and “information content.” Can you please give your criteria for these two things so I can begin contesting them? I’m concerned you communicated no “ordinary sense” information in your preceding comment and there was zero information content, and I’m trying out your style where that preempts the rest of the conversation flow.
The statements “gives” you information but that doesn’t “count” as you “getting” information. Furthermore the “low-information” statement mysteriously gives you information, yet not quite enough information to count as not a “low-information” statement. Okay, so this isn’t about communicating information, it’s about communicating information with a twist—it also has to count. If your interlocutor communicates successfully but it doesn’t count, you’re allowed to make a definition challenge, where they have to provide a set of criteria you’re allowed to attack. They then try to defend their criteria while you try to attack it, and if they defeat the definition challenge you’re allowed to move onto the next message in the initial communication.
Okay so why would anyone ever agree to play this game? It sounds completely not fun and like it’s optimized for the most uninformative things you could talk about.
Oh I see. After saying “I don’t like chemicals in my food” I understood something like, this person prefers organics. Are you not able to surmise this? If you’re not, then you’re definitely gaining less information when talking people than I am. I can generally communicate with people even if they use “chemical” imprecisely.
That’s not knows what they’re talking about, that’s has talked to people who sound like they know what they’re talking about. The epistemic status is clear about this so I’m not knocking OP. “Epistemic status: thing people have told me that seems right.” This is actually what taking hearsay from insiders seriously looks like.
Key point: minimalism the design aesthetic, and minimalism meaning actually having less stuff, are opposites. Minimalist design means you highlight the one function it does very pleasingly. Minimalist stuffhaving means you have fairly cheap-looking multipurpose tools.
Ironically the most important domain this applies is storage containers. Your path to decluttering succeeds if your first instinct is to grab random boxes/cartons/bags to start storing stuff, and it fails if you start by buying new containers.
Marie Kondo is a very good book, you should read it.