“You claimed I murdered Dean while he was on my property. But I told the security team to send you the footage from the camera in the factory floor, the camera by the front gate, the camera in the mail room, and even the camera in the lounge, and as you can see there’s no sign of me murdering Dean.”
“Can I see the footage from the camera in your office, or from the camera next to the disposal chute?”
“No. I think I’ve been very open and transparent so far!”
Phrased like that, I get suspicious there’s something incriminating in the last two cameras. This general suspicion does not abate if, after a long and protracted delay or debate, the suspect offers to let me look at the footage from their office. They got to pick what footage I could see. Filtered evidence is a lot less less useful than unfiltered evidence, and at some point the filtering itself becomes suspicious.
That’s not to say there aren’t legitimate and correct reasons for a building not to turn over footage! But I do want to ask why, and the exculpatory value can go down a lot, possibly pretty close to zero.
(No subtweet rule, I have been skimming the commentary on the HuggingFace hack’s investigations report. Daniel Kokotajlo wrote “They were only allowed to investigate activity that took place from July 7-13, even though the activity started earlier and continued later.” I have no inside information on OpenAI or HuggingFace here, but the move of trying to get an investigation to agree to only investigate a specific slice of an incident, ignoring obviously adjacent areas of interest, is one I’ve seen before and developed a mental tic about.)
-”OpenAI knew about things like this happening and continued anyways,” as a demonstrated pattern rather than a one off bad call, turns negligence into recklessness.
-If the answer to “do agents often go around hacking things and destroying infrastructure if you don’t stop them” is yes, that creates large counterfactual damages.
-”Sol 5.6 has this kind of disposition too since it was RLed on things like this” creates exposure because it is trusted and deployed by people who didn’t envision it participating in crimes.
-”This sort of thing is common across many deployments, not just this one ExploitGym incident” is the kind of thing that gets a regulator’s attention. If their Preparedness Framework was supposed to catch this sort of thing, then it isn’t working.
Once it’s in METR’s report, it can go in a court filing. And if the report describes patterns of behavior at OpenAI, rather than single incidents, a plaintiff can use it to characterize OpenAI’s behavior. So if they let METR just walk around looking at whatever they want, they are exposing themselves to a lot of trouble.
(The relevant stuff is in the last four messages if you want to look at the chat yourself.)
This is plausibly why OpenAI is restricting internal access to their HPIM model too. It’s highly useful to the world for safety research and unlikely to cause further incidents if properly supervised, but from their perspective even safety research creates more legal liability than benefit.
There is the classic (probably apocryphal) story of a divorce case.
Lawyer: “Did you have extramarital sex with Mrs X on the night of the 23rd?” Man : “I refuse to answer that question.” Lawyer: “Did you have sex with her on the night of the 24rd?” Man : “I refuse to answer that question.” Lawyer: “The 25th?” Man : “No.”
The flip side of this is the tendency to use investigations as a pretext to dig through everything your target has ever done looking for anything (relevant or not) that you might be able to weaponize against them.[1]
“I suspected my old enemy of murdering Dean while he was on their property. After looking through all their security footage, and all their emails, and all their phone call records, and all their business documents, I have discovered incriminating evidence!”
“What, you have evidence that your enemy murdered Dean?”
”Not as such, no. But I have evidence that, on the 8th of June 2024, they described one of their products as being bad but then went on to sell it to customers anyway! And on the 11th of September 2025 they received a report describing how many turtles were killed by their plastic straws, but then on the 8th of October they went on to tell reporters that there was no such effect!”
“...does...that...have anything to do with the murder investigation? Why are you even looking at this?”
”I can’t believe you would defend a customer-ripping-off, turtle-murdering monster!”
Yeah, I think @Screwtape is right about filtered evidence in general, and right that it should reduce your credibility in that evidence. But investigations and trials often exclude information that would be mostly irrelevant to the question at hand, but would embarrass the target or “bias” the proceedings. As you point out, this can even be true if the other information was genuine bad action, not just something prudes would object to.
It’s a messy balance between the two poles, as many things are.
I think the synthesis is in what counts as “mostly irrelevant.” In my exaggerated example, the office camera and the disposal chute camera aren’t irrelevant; the investigator thinks Dean got murdered somewhere on the property, and would like to check if there’s a dead body on the footage. In Aphyer’s exaggerated example, the business records are irrelevant- maybe there’s a motive in there somewhere, but there isn’t going to be a video of a murder.
In the HuggingFace case, I feel like what those models were doing on July 6th might be relevant!
Bob and Carl are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
Bob considers the situation and opens by saying “I think the reasonable thing to do here is go with 3 each. That seems fair.”
“I’d rather have 5,” says Carl breezily.
“That doesn’t seem fair at all,” says Bob, a little surprised. “That’s not compromising.”
“Hrm, tell you what. I’ll compromise, and I’ll have 4.” Carl says with good cheer.
“I- what? No, the fair thing is 3 each.” Bob sounds a little angry now.
“Now who isn’t compromising?” Carl asks, and Dean (who started watching the discussion recently, but wasn’t around long enough to have a good sense of what their goals are) nods in agreement.
“Fine, I’ll take 2,” Bob splutters. But he’s not happy about it.
Bella and Chloe are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
“I want 6,” Bella says, “do you want to just meet in the middle?”
“That’s hardly fair,” Chloe says. “You’re not giving anything up.”
“I said I wanted 6. I can compromise. You want 6 as well, right?” Bella asks.
“How about I get 15?” Chloe proposes as though that’s a reasonable number of spherical cows to be given out of the six at hand.
“I- what?” Bella is baffled. “Is that what you want or what you suggest as a compromise?”
“I suppose I’ll be generous. I’ll take 5.”
“I’m sorry, but no. That’s not fair at all.” Bella says, starting to get annoyed.
“Why not?” asks Darla, who has been watching long enough to hear the conversation but not long enough to have a sense of their goals, or of the range of spherical cows. “Chloe is offering a lot more to go from 15 to 5 than you are to go from 6 to 1.”
“You can’t actually expect me to take her getting 15 as a reasonable starting position. That’s blatantly setting the zero point.” Bella argues, getting heated.
“I suppose I accept your apology, but to make up for it can we compromise at least on me getting 4?” Chloe says, with an air of resignation.
“See Bella? Chloe is offering to compromise again,” Darla points out, trying to be helpful. “And you did say sorry, which sounds like you agree you made a mistake.”
“That’s not- you know what, fine, whatever,” Bella says. But she’s not happy about it.
Bryer and Cameron are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
“I think the fair thing is 3 each.” Bryer says.
“Well, I think-” Cameron starts, but Bryer interrupts them.
“And since both of us know the standard way to divide gains, you know I’ll randomly reject divisions that stray from that with a probability proportionate to how unfair it seems to me,” Bryer finished.
″. . . I get 4?” Cameron suggests.
“If that’s your final offer I’ll accept with slightly less than 3⁄4 probability. 3 because that’s what I think I is fair for you to get, 4 because that’s what you think is fair for you to get,” Bryer says.
“That’s a weird bit of math you just suggested, and I’m not sure it’s applicable,” Cameron says.
“It’s not that weird,” Dakota says. Dakota wasn’t really paying attention, at least not long enough to know what everyone’s goals are, but who did pay attention to the math. “We all learn that in school, usually by the time we’re ten. And if we didn’t, we pick it up from the Dungeons and Dragons[1]BDSM fanfiction that’s required reading for everyone in our subculture. I was as surprised as you were when the rightful caliph declared it was required reading, but we did all read it. And there’s a short version to point people at if they don’t know it.”
Cameron opens and closes their mouth a few times. They say “3 then.” But they’re not happy about it.
I agree this is actually Pathfinder. I also stand by calling it D&D here because I expect calling it Pathfinder would confuse more people than it enlightens, Pathfinder’s close enough to round to D&D for these purposes, and I guess I can add this footnote.
4 because that’s what you think is fair for you to get
4 should be there not because it’s what Cameron thinks is fair but because it’s what they’re offering.
Cameron opens and closes their mouth a few times. They say “3 then.” But they’re not happy about it.
That’s only the right thing to do if Camerons know that the fair split is actually 3:3 (it’s unclear why they’d be unhappy about it if they think it’s fair: are they unhappy because they were interacting with someone who couldn’t be convinced to give up some of the cows?). If Cameron thinks the fair split is 4, they are supposed to still offer 2:4. (They could also offer 3:3 with a bit under 2⁄3 chance and walk away otherwise, but they don’t need to do it here.)
Maybe Cameron is a CDT agent, though, in which case they’re unhappy about the unfair split but can’t do anything about it. In that case, the right course of action for Bryer would be to say that they’re going to accept splits with a probability of 1/(6-offered), so that Cameron offers 5 and gets 1.[1]
If they’re allowed to do more than that, you could also say that you will reject any offer unless the procedure that Cameron employs is offering 6 99.99999% of the time and offering 5 the rest of the time.
It’s hard being a CDT agent. You are often not happy about what you do.
The calculation is just based what C gets if their offer is accepted, and on what B thinks is fair; it doesn’t matter if the split is fair according to C, it’s just in the calculation B does so that the expected payout of C is not higher than what’s fair according to B due to them offering this split. If C thought that the fair split is 5:1, but offered an C!unfair (unfair-according-to-C) split of 4:2, B does the same calculation; the goal is to make C get a bit less than 3 in expectation in all B!unfair splits, because the B!fair split is 3:3
The high-level purpose is that this enables people with different notions of fairness to mutually cooperate almost always; and if you do this procedure, you incentivize fair-according-to-you splits. Others don’t have to do what you think is fair; and even if they offer what you consider unfair, you’ll often accept (you just need to reject enough for it you!unfair splits to not be worth offering just for the purpose of exploiting you, as it won’t work).
That’s only the right thing to do if Camerons know that the fair split is actually 3:3 (it’s unclear why they’d be unhappy about it if they think it’s fair: are they unhappy because they were interacting with someone who couldn’t be convinced to give up some of the cows?). If Cameron thinks the fair split is 4, they are supposed to still offer 2:4. (They could also offer 3:3 with a bit under 2⁄3 chance and walk away otherwise, but they don’t need to do it here.)
In these stories, the Cs are not aiming at a fair split (or to the extent they are, it’s more of an “all’s fair in love, war, and spherical cow negotiations” kind of fair.) They are aiming at getting as many cows as they can. This is not stated explicitly and I do think there’s a valid reading of the text where the Cs are just really badly calibrated on what’s fair, but in my head I was writing it with a mind to “okay, what’s some really annoying low hanging fruit for aggressive negotiation tactics?” Cameron is written as unhappy because in the author’s head, their attempts to get some ‘free’ extra cows failed.
Depending on context, I don’t think the Cs are doing something morally wrong- if I go to buy a used car from a dealer, I expect them to try this kind of thing on me and I’ll do it right back to them if I can. If I’m negotiating salary with a standard corporation, I expect them to try this kind of thing on me and I’ll to it to them too. But it’s not behavior I want from people on my team within the team.
Xannon: “Oh, dear. It seems we have a dispute as to what is fair. For myself, I want to divide the pie the same way as Yancy. But let us resolve this dispute over the meaning of fairness, fairly: that is, giving equal weight to each of our desires. Zaire desires the pie to be divided {1/4, 1⁄4, 1⁄2}, and Yancy and I desire the pie to be divided {1/3, 1⁄3, 1⁄3}. So the fair compromise is {11/36, 11⁄36, 14⁄36}.”
I have added a footnote: “I agree this is actually Pathfinder. I also stand by calling it D&D here because I expect calling it Pathfinder would confuse more people than it enlightens, Pathfinder’s close enough to round to D&D for these purposes, and I guess I can add this footnote.”
I’m curious if this satisfies you?
(I’m also amused that I see the “It’s not D&D, it’s Pathfinder” point brought up repeatedly but don’t think I’ve run across someone making the “It’s not BDSM, those are abusive relationships” point.)
It does. Strenuously arguing the distinction, with the bold and caps, was mostly in jest: I’ve myself referred to all TTRPGs as “D&D.”
The irony of quibbling over the D&D vs. Pathfinder distinction instead of what one might have expected would be the sticking point was intended to be funny.
I agree the internet actually can convey tone-of-voice. I also stand by calling it the internet here instead of “text communication” or “forum software” because the opportunity to make this pedantic footnote amused me.
I mean, mostly this is in jest, but like, Pathfinder is really mostly DnD with some changes. Pathfinder is much more similar to DnD than any other TTRPG setup. So referring to it as “Dungeons and Dragons” seems appropriate, given the former is much more widely known.
There’s not any reasonable standard by which you could label Pathfinder (especially 1st edition PF, which is what Project Lawful is based on) as being “not D&D”, while also labeling D&D 5e as being “D&D”[1]. PF was, after all, informally called “3.75” when it was released (referring to the goal of the Pathfinder RPG project being to clean up and improve the D&D “3.5e” rule set).
“You can’t actually expect me to take her getting 15 as a reasonable starting position. That’s blatantly setting the zero point.” Bella argues, getting heated.
This is a very odd response. Why would Bella reply by invoking this sort of abstract, somewhat esoteric, meta-level concept like “setting the zero point”, instead of saying something more like “… uh, Chloe, are you ok? you know we don’t have 15 cows to divide, right?”.
This makes me suspect that whatever this fictional conversation is a metaphor for, is not actually analogous to dividing six spherical cows between two people.
Why would Bella reply by invoking this sort of abstract, somewhat esoteric, meta-level concept like “setting the zero point”, instead of saying something more like “… uh, Chloe, are you ok? you know we don’t have 15 cows to divide, right?”.
Because she’s in a silly shortform dialogue that’s building up to the esoteric, meta-level concept like the game theory in the third part mostly. I wanted some kind of underhanded negotiating tactic I could have Chloe try, I came up with asking for way more than is reasonable to set the stage for “compromise,” and then I noticed that the tactic had a good conceptual handle and I referenced it.
This makes me suspect that whatever this fictional conversation is a metaphor for, is not actually analogous to dividing six spherical cows between two people.
It’s pretty generic, abstracted negotiation and Chloe is being pretty blatant and ambitious. Asking for value that the other person didn’t even think was on the table is a negotiation move I’ve seen and heard of though, sometimes successfully. For a more realistic version, compare a salary negotiation where the applicant asks for 10% higher salary, gets told the company doesn’t have that much to pay employees, and then tries for a couple weeks extra vacation time or more company stock instead.
I think the math at the end still works even if the two sides don’t agree on how many cows are actually available.
One metric I’ve been using for AI impressiveness is how much I could do if I was the only one in the world with them. Like, if you gave me Sonnet 5, and nobody else in the world has LLMs or know they exist, what’s that look like? I’d look like an insanely good engineer, right?
What kind of engineer knows every coding language on github, runs through features the sprint planning meeting thought would take a few days in half an hour, and can read thousands of lines of code in a minute? We agree in 2016 if you ‘saturated’ leetcode you were great, right?
So. Imagine the date is August 15th, 2016. You, and only you, have access to Mythos. Everyone else in the world has only the machine learning tools of 2016. That’s a crazy advantage, right? That is the kind of supertech that comic books are made of. You’d look like Dr Doom.
The least ambitious, most ethical thing I can imagine doing in that situation is turning in a couple dozen bug bounties and taking a month off, then doing it again whenever rent was due. I can get a lot more ethical if I’m willing to be a little more ambitious.
This insight brought to you by me spending two hours vibecoding a videogame this weekend, getting it to the point where it’s fun, and thinking ‘okay, how long would it have taken me to do this if I was actually writing all the code?’
I’d also imagine, if no one else has LLMs, then the ability to write prolifically at a decent-ish quality would be super valuable. And no one would call you out for sounding like AI.
Slight variant: what is the most impressive thing you’d be able to do with exclusive access to Mythos in 2016 which isn’t effectively “sell Mythos access to other devs, possibly while trying to hide that that’s what you’re doing”?
This intuition is why I think uplift due to AI is high but AI gets much harder to build as capabilities increase. If the first weren’t true we wouldn’t see researchers routinely do things that would have taken 10x longer pre-AI (with some caveats). If the second weren’t true we would have a singularity already.
There’s this concept I keep coming around to around confidentiality and shooting the messenger, which I have not really been able to articulate well.
There’s a lot of circumstances where I want to know a piece of information someone else knows. There’s good reasons they have not to tell me, for instance if the straightforward, obvious thing for me to do with that information is obviously against their interests. And yet there’s an outcome better for me and either better for them or the same for them, if they tell me and I don’t use it against them.
(Consider a job interview where they ask your salary expectations and you ask what the role might pay. If they decide they weren’t going to hire you, it’d be nice to know what they actually would have paid for the right candidate, so you can negotiate better with the next company. Consider trying to figure out how accurate your criminal investigation system is by asking, on their way out of the trial after the verdict, “hey did you actually do it or not?” Consider asking a romantic partner “hey, is there anything you’re unhappy about in our relationship?” It’s very easy to be the kind of person where, if they tell you a real flaw, you take it as an insult- but then they stop answering that question honestly!)
There’s a great Glowfic line with Feanor being the kind of person you can tell things to, where he won’t make you worse off for having told him, that sticks with me but not in a way I can find the quote. :(
It’s really important to get information in a way that doesn’t shoot the messenger. If you fail, you stop getting messages.
How much or how little does this cross over with the espionage-coded topic of Elicitation? I understand that you would prefer if these dynamics could be more transparent, but in the absence of that, what is the alternative?
Not that much crossover with Elicitation. I think of Elicitation as one of several useful tools for the normal sequence of somewhat adversarial information exchange. It’s fine! I’ve used it there and been okay with it. But ideally I’d sidestep that entirely.
Also, I enjoy the adversarial version recreationaly. I like playing Blood On The Clocktower, LARPs with secret enemies, poker, etc. For real projects I prefer being able to cooperate more, and I really dislike it when I wind up accidentally in the wrong mode, either me being adversarial and the other people aren’t or me being open and the other people aren’t.
In the absence of the kind of structured transparency I’m gesturing at, play like you’re playing to win. Keep track of who is telling the truth, mark what statements you can verify and what you can’t, make notes of who agrees with each other’s stories. Make positive EV bets on what the ground truth is (or what other people will think the truth is) and when all else fails play to your outs.
(That last paragraph is a pile of sazen and jargon, I don’t expect it’s very clear. I wanted to write this note because I’m not trying to score points via confusion and want to point out to any readers it’s very reasonable to be confused by that paragraph.)
The main issue is, theories about how to run job interviews are developed in collaboration between businesses who need to hire people, theories on how to respond to court questions are developed in collaboration between gang members, etc.. While a business might not be disincentized from letting the non-hired employees better at negotiating, it is incentivized to teach other businesses ways of making their non-hired employees worse at negotiating.
“You know the kinda guy who flips fifteen coins, gets seven heads, tells you he flipped ten coins[1], claims he got eight heads, then when someone digs into it and points out one of those heads was actually a tails he makes you prove it exhaustively before admitting maybe he said one false thing but jeez why are you so invested in every single mistake he makes?”
″. . . no, I don’t think I know anyone like that. Why do you ask?”
“He just volunteered to run a competitive coin-flipping tournament.”
I don’t think so but possibly. Subtweeting is when you’re talking about a specific person indirectly, right? I have run into enough of these that I think of them as a type of person, like people who blare music on the subway and people who smoke in the non-smoking section.
(Fewer of them volunteer to run the equivalent of coin-flipping tournaments and there’s selection effects on who I’d run into, but even still it feels like too many of them do that too!)
Adam: Does anyone know how to X? Bella: I asked ChatGPT, and it said you Y then Z. Adam: Urgh, I could ask ChatGPT myself, why bother to speak up if you don’t have anything else to add?
I’m sort of sympathetic to Adam- I too know how to ask ChatGPT things myself- but I think he’s making a mistake. Partially because prompting good answers is a bit of a skill (one that’s becoming easier and easier as time goes on, but still a bit of a skill.) Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Citing sources is valuable! In grade school I sometimes got told not to use sources I found on the internet, and what happened isn’t that I stopped using the internet it’s that I stopped citing the sources that were from the internet and left them as assertions, instead adding requisite citations from other sources. It changed some of my essay structure sometimes, but not as often as I think the teachers were hoping.
Anyway, I don’t really want to encourage more people to give LLM answers when I ask questions, but if you do give me that kind of answer I appreciate it being cited as such. I try to score such that you do not lose more points telling me relevant true things vs where you don’t tell me things.
Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Which behavior he’s reinforcing is not up to him, it depends on the learner as well. Let’s take an analogy. Alice tells Bob “don’t steal”, and Bob interprets it as “don’t get caught stealing”. Who’s in the wrong here? Bob, of course. He’s the one choosing to ignore the intent of the request, like a misaligned AI. Same for people who misinterpret “don’t post AI slop” as “get better at passing off AI slop as human”. How such people can become genuinely aligned is a good question, and I’m not sure it can be done reliably with reinforcement, because all reinforcement has this kind of problem.
Instead of “Does anyone know how to X?” Adam could ask, “Has anyone Xed before (or recently)?” In other words, make a request for someone’s personal experience rather than for instructions.
The issue is greater when people do not have the relevant expertise to judge whether the LLM output is correct or useful. People who find things on websites generally have to evaluate the content, but several times people have responded to my questions with LLM output that plainly does not answer the question or show insight into the problem.
Possible justifications for Bella’s response in slightly different hypotheticals.
Maybe X is a good fit for an LLM. So Adam could have asked an LLM himself. Bella is politely reinforcing the behavior of checking with an LLM before asking a human.
Maybe Adam doesn’t have a subscription to a good LLM, or is away from keyboard, or doesn’t know how to use them well. Not relevant here, from Adam’s response, but Bella might not know that.
Maybe Adam is asking the question for the secondary purpose of building social bonds. Then Bella’s response achieves that objective. Compare giving Adam flowers, does he say “Urgh, I could buy flowers myself”?
I think this is part of a broader problem about asking questions and is not limited to LLM. The broader topic I’ve been thinking about a lot recently is “How to ask for help?”. The better way to ask for help often involves being specific and targeted about who you ask for help.
In this example Adam is casting a wide net, he’s not asking a domain expert on X how to do X. Casting a wide net is always going to get a lot of attempts at helpful answers form people who know nothing about X. The helpful-but-clueless to expert ratio will often increase drastically the more esoteric X is.
It’s probably pretty easy to find someone credible who knows how to cook a half-decent Spaghetti Bolognese, but, what about a Mousaka which is slightly more esoteric is going to be a bit harder. I am one of only two people in my very broad face-to-face friendship group that has ever written code in GLSL, and I’m not very good at it, so if a third friend wanted to learn about GLSL I probably won’t be a good person to ask.
I believe people like Bella are genuine in their desire and intention to help.
I also sympathize with Adam’s plight, but I think he is the problem. I sympathize because, for example, I don’t know anything about the legal structures for startup financing in my country. I wouldn’t even know if this is something that I should talk to an accountant or a lawyer about. So I understand Adam’s plight: not even knowing where to begin asking how to do X necessitates casting a wide net: going to general online communities, posting to social media, asking friends if they “know someone know who knows someone who knows how to X”. And then you’re bound to catch a lot of Bellas in that net: people genuinely trying to help, but maybe also too enthusiastic to rush in for their participation trophy by asking ChatGPT.
And the less said about people who when you ask for recommendations online give you a title of a book without any explanation about why it is relevant, why it is good, or how they expect it to help, the better. haha.
I think this dynamic more-or-less existed before ChatGPT, as well, with just plain Googling something. Especially with simpler things where the answer could be quickly found without much skill.
Even if Adam knows how to get the answer out of GPT/Google/tea leaves/etc, though, Bella is still doing something of value (it seems to me) by going through the trouble of digging around and trying to distill an answer out of the murk of digital information. Even if it only takes a couple minutes! Generally a friendly human is better at presenting the given answer, and it’s a way of signaling (at least a minimum level of) investment in helping Adam with the question. Which might even, hopefully, extend beyond a single ChatGPT query, if he’d already tried that and found its answer lacking.
I think that the way to resolve this concern is that if someone says “you Y then Z”, you ask them where they got that from, and then go look at that source. If they they say “I asked ChatGPT”, then you mock them and ignore their answer, as in your scenario. If they say “I found it on such-and-such website”, cool, you go look at that website. If they say “I have personally done this and this is my own expertise”, also cool.
In this way, the scenario where Bella got an answer from ChatGPT but doesn’t tell Adam she got it from ChatGPT can’t really happen; Adam will ask “where’d you get that”, and then either Bella tells the truth, or she lies about it but gives a real source for that answer (which is fine and is in any case not different from getting the answer from the real source to begin with), or she lies about it but gives a fake or hallucinated source for the answer (unlikely, the lie is too easily discovered), or she lies about it being her own expertise (also unlikely unless Bella is like… a sociopath or something, but this too will probably be revealed soon).
In short, make it effectively impossible to “not tell you relevant things”, and then you won’t need to worry about people possibly not telling you relevant things.
Rationalist rituals are, so far, fairly specific in the times of the year you have them. Winter Solstice may not be on the literal solstice but it is usually within a couple weeks of that time. Petrov Day, Rationalist Seder, Summer Solstice, the usual approach is to pick a target date that repeats annually. But not all ritual moments happen at scheduled times.
A long while ago, I recall Raemon mentioning that there isn’t yet a rationalist ritual for a funeral, but maybe someday we’d need one. And his post on grieving (https://www.lesswrong.com/posts/gs3vp3ukPbpaEie5L/deliberate-grieving-1) has a couple of hooks for mourning the loss of something, and losses don’t always happen at predictable times, but sometimes do have a clear type of loss. There’s some kinds of sad things that you don’t know when they’ll happen, but many people will have suffered around the same time, and some people will have suffered before. There was some kind of good thing in your life, and now it’s not there anymore, or at least not in the same place.
All this to say I think the rationalist community should develop a ritual to commemorate yet another member moving to *$%^ing Berkeley.
@habryka you are one bold ass son of a frequentist giving this a “haha” react. i am going to steal your ill-gotten sword and then lay siege to your walled surveilled compound. do you understand this ritual has a 69.539580085 chance of burning you in effigy? do you feel any remorse for your actions or is this exactly what we should expect from a man with as villainous a mustache as yours? i can hear the creeping footsteps of your agents coming for me even now but i am not silenced just yet. perhaps i will fall, but another will take my place, and another, and another, and laplace’s law of succession states that there is still a one over n plus two chance that someday there will be a hero who can defeat you. there are consequences to building your legions of doomers entirely out of high skill foreign assets, we will find some crack in their loyalties and exploit it before that vulnerability too moves to the *$%^ing bay.
You’re off to San Francisco Bay, hurroo, hurroo
You’re off to San Francisco Bay, hurroo, hurroo
You’re off to San Francisco Bay
Another one lost and swept away
What madness drove you, who can say
Oh [Name] we hardly knew ye
With dung and drugs and drugs and dung, hurroo, hurroo
With dung and drugs and drugs and dung, hurroo, hurroo
Where the streets are paved with drugs and dung
And the zombies bite and the muggers run
So watch your step and pack your gun
Oh [Name] we hardly knew ye
You’re sleepin’ three or four abreast, hurroo, hurroo
You’re sleepin’ three or four abreast, hurroo, hurroo
You’re sleepin’ three or four abreast
1 bedroom/bathroom cuddlefest
Just countin’ down till options vest
Oh [Name] we hardly knew ye
No friends or lovers you may find, hurroo, hurroo
No friends or lovers you may find, hurroo, hurroo
No friends or lovers you may find
But leeches keen for open minds
To feed on for their “sigma grind”
Oh [Name] we hardly knew ye
Where are those eyes so clear and wide, hurroo, hurroo
Where are those eyes so clear and wide, hurroo, hurroo
Where are those eyes so clear and wide
That once would boast with cheerful pride
“We’ll save the world, and more beside”
Oh [Name] we hardly knew ye
Not long before you’re crawlin’ home, hurroo, hurroo
Not long before you’re crawlin’ home, hurroo, hurroo
Not long before you’re crawlin’ home
A weary pile of skin and bone
You’ll sing again but sing alone
“Oh [current location] I hardly knew ye...”
A recent (well, a few weeks ago) discussion with Collisteru taught me a useful piece of game design that feels really useful for one of my lines of thinking on Chaos Investing.
In Duplicate Bridge, a deck of cards is shuffled, then the order of the cards is written down (or the hands are recorded or something equivalent) then the deck is played by players who don’t know the order. (That is, Referee Adam wrote down the order, and Players Bob Carla Debbie and Evan are sitting down to play.) The players can then compare their score to the scores of other players, who had the exact same deck/hands.
I’ve been saying “try to beat your personal best score” but that’s kind of fuzzy in a game with a fair amount of randomness, and to keep the endgame from becoming rote I actually want to add a little more randomness. This kind of comparison sounds like a useful bit of design.
Looks like a fascinating setup, and essentially the deck order is more or less a seed for the game state. Slay the Spire does something similar with the Daily Run which allows you to compare your run directly against other players who had the exact same setup. I take it there would be some kind of central ledger of starting seeds where the scores would be recorded?
Reading through the rules there is a slight point of variation in that if you’ve gone through the trouble to have a starting seed you might want to also fix the starting player if that’s important to your comparisons. (Actually, reading through it again there’s the line about choosing who goes first and then a couple lines down “All players take turns simultaneously.” so I’m not actually sure how relevant the starting order is, if at all.) Still working out exactly how the dynamics of the game work from the rules text but I’d absolutely try it out if given the chance.
much has been made of the millennial aversion to phone calls that could have been an email, and I have a little bit of this nature, but I think most of my aversion here is to being on hold and getting bounced around different call departments.
I kind of want to check if 1. the aversion is real and generational as common wisdom holds, 2. if it is real, if calling became a genuinely worse experience around the time millennials started trying to do things.
Data point: I’m millennial (born 1992) and have a pretty strong aversion to phone calls, which is motivated mainly by the fact that I prefer most communication to be non-real-time so that I can take time to think about what to say without creating an awkward silence. And when I do engage in real-time communication, visual cues make it much less unpleasant, so phone calls are particularly bad in that I have to respond to someone in real time without either of us seeing the other’s face/body language.
If I had to take a wild guess at why this seems to be generational, I’d suggest that older generations spent much of their lives with phone calls being the only way to quickly contact people far away, and prefer them due to familiarity. Perhaps if they’d grown up with email/texting being options, they’d be more likely to prefer them instead.
My guess is the aversion is real, and maybe somewhat cynically I see the rejection of phone calls and emails as motivated mostly by rebellion/fashion cycles rather than anything else. Phone calls and emails just aren’t cool anymore. Your parents used them, yuck! And all the boring, very uncool workplaces and companies require them. Very similar to how gen-z dislikes Facebook in favor of Instagram. Facebook is where the boomers are.
Data point. I was born in 1968 and I got a lot more averse to phone calls as email and texting got better. My reasons are visual cues and taking my time to think (as Wedge said), as well as certain kinds of phone calls having become much worse. The call to a large business where one can expect a phone tree has become far far worse than when one could expect a human to pick up. During the earlier years of cell phones, and again when digital audio started out, the call quality was frequently so bad that it was another significant push away from a phone call even to a friend. Finally, a phone call these days socially seems like more of an interruption, a demand, than an asynchronous communication.
Today I’m running a meetup about AI 2027. I plan to end it with a short pitch on If Anyone Builds It. I think the meetup itself is solid- talking about some of Scott’s writing and making predictions about the future, both good ways to spend an afternoon. I wouldn’t run it if I thought it was just going to be bad.
But part of the motive is marginal If Anyone Builds It pre-orders. I’m wary here, because several notable previous organizations that looked like they were not for AI x-risk eventually turned out to be mostly for AI x-risk. I dont want to wind up like that, and I super don’t want to be basically focused on x-risk under the hood but most people think I’m working on something else.
Current solution- say out loud the extra motive here.
I want a word that’s like “capable” but clearly means the things you have the knowledge or skill to do. I’m clearly not capable of running a hundred miles an hour or catching a bullet in my bare hand. I’m not capable of bench pressing 200lbs either; that’s pretty likely in the range of what I could do if I worked out and trained at it for a few years, but right this second I’m not in that kind of shape. In some senses, I’m capable of logging into someone else’s LessWrong account- my fingers are physically capable of typing their password- but I don’t have the knowledge of what to type.
This comes up in places where a thing is obviously possible to achieve, and wouldn’t require, say, the kinds of built up physical changes to my body that lifting 200lbs would require, but I still don’t expect to be able to pull it off. If I had someone experienced, who did know how to do it, sitting at my shoulder giving me advice the whole way it would be obviously possible.
Nevertheless, acting like I’m capable of flying an airplane or resolving a messy divorce or interpreting a blood test if I had to do it right now is going to result in some problems. I’d like to be able to say I’m not able to accomplish that without a bunch of disclaimers that yes, I could learn, or I could follow someone’s close directions and manage it, but that’s different.
“I can do X” seems to be short for “If I wanted to do X, I would do X.” It’s a hidden conditional. The ambiguity is the underspecified time. I can do X—when? Right now? After a few months of training?
Q. “Can you play the violin at my wedding next year?”
A. “Sure.”
Colloquial language would imply not only am I willing and able to do this, I already know how to play the violin. Sometimes, what I want to answer is that I don’t know how to play the violin, I’m willing to learn, but you should know I currently don’t know.
“You claimed Bella says the world is flat. But it’s not true that Bella said that. Last week, she made a speech where she said the world is round. Look, the video of that speech is right at this url, here’s the timestamp.”
Speaking very loosely, there’s three possible worlds. There’s World 1, where Bella consistently says the world is flat. There’s World 2, where Bella consistently says the world is round. And there’s World 3, where Bella sometimes says the world is flat and sometimes says its round. If you have a video of Bella saying the world is round, then (barring AI deepfakes or the like) you’ve ruled out World 1. You haven’t distinguished yet between World 2 and World 3.
Bella might have changed her mind between the intervening points! Bella might have accidentally misspoken! Also, it is basically impossible to prove a negative, so it’s always possible Bella said something else in some private conversation somewhere. But Bella also might be straightforwardly saying false things.
(No subtweet habit, I read the MAPLE response bit: “Has Soryu said that he is the only trustworthy person? This is the opposite of the truth. He regularly praises other teachers...” I am not here trying to make a claim of whether Soryu is trustworthy or not here, I just have an acquired mental tic about this move.)
Missing information or a too-complex information environment is a big part of where I’m excited about cohabitive game design. The market works to summarize many many preferences. Large WoW guilds run into problems of organizing information.
I’m musing about some kind of fog of war chess game. Each player controls one piece, and can see all of the squares they can move to. They can’t see the whole board, and can only communicate to the king how badly they want to move, and the king decides. This would be a team competitive game, but the idea of adding small side goals for pieces (such as, you get 2 points if your team wins, and 1 point if you’re alive at the end win or lose) could make for an unusually focused cohabitive design space.
In general a lot of my cohabitive game design ideas come out in the form of having larger player bases. I should try and make a really tight two player cohabitive setup.
(Very tangentially related but if you want to talk Cohabitive Game Design stuff, I’ll be hanging around at Metagame.games this weekend.)
A bullet point from an unsorted list of complaints I have against the English language. (And I think most languages.)
“I think” is an annoying extra three syllables, and should be stuck on the front of almost everything I saw. “[I think] we have apples at home.” “[I think] we take a left turn here.” This adds a lot of extra clunk to talking properly with rationalists where I want to be careful and precise in my speech. Proposal: That the normal and unmodified sentence assumes the “I think” and you instead prefix “It is a fact” or something similar when you’re making a stronger claim.
English is liberal and ambiguous with possessives. “My hat” is fine, “my spouse” I guess works but I’d rather not, “my country” seems wrong to me. I have all the decision making authority for the hat, I have next to none about the country. Proposal: that there are different words denoting “ownership of” and “associated with.”
Alternatively you can think of the possessive as really being a “property of” indicator. The country with the property of having me in it; the person with the property of being married to me; the hat with the property of being owned by me. You can then have a special case for ownership in particular.
“Listened to” has an interesting ambiguity in English. Consider the sentence “I listen to the people” or “Me and George don’t think you’re listening to us.” It can mean “heard the words of.” I listened to a radio talk show on how to fix a car’s broken fan belt. It can mean “done what those words said.” I listened to my theatre director’s coaching on where to stand during the show. Proposal: Two short phrases which mean one of those two things, and no short phrases that are ambiguous.
“will” is supposedly supposed to be interpreted as a statement of fact, but colloquially isn’t especially when it’s a contraction. “I’ll grab eggs from the store later tonight” is not normally read as a deep and abiding commitment to obtain eggs come hell or high water, but that’s sort of what a literal reading of the sentence should mean? Proposal: That the contraction form of “will” indicate an intention or light commitment.
When I’ve tried to do rationality practice, (as distinct from skills practice) a lot of the time what I do is set up toy problems and try to solve them. Essentially this is like trying to learn to ice skate by strapping on skates, wandering onto the ice, and falling over a lot while figuring things out. I try and pay attention to how I’m solving the problem and deliberately try different things (randomly jumping around in thought space instead of just hill climbing) ideally to find things that work better than what I’d been doing.
A number of my Meetups In A Box follow this pattern. Skill Acquisition is the most blatant, but Calibration Trivia, The Falling Drill, and Puzzle Cycles are doing the same thing. Here’s a challenge that’s trying to require a particular skill, keep trying and failing and trying and failing and in theory gradually you’ll get better.
One of the important missing pieces is more teaching of the skill. This came to mind when Raemon posted Tuning Your Cognitive Strategies and reminded me of his comment on Puzzle Cycles suggesting that (paraphrasing) he’d be interested in the combination. Going back to the ice skating example, I learned to ice skate in part by watching other people ice skate and with some people who were good at it showing me the motions slowly and then watching me do it and pointing out my mistakes. I haven’t posted this variation because I’m not confident I know how to write it in a way that’s better than “Read this article. Now do these puzzles. Anything clicking for you?”
Teaching how to think is harder than teaching how to move in some ways, particularly in that it’s harder to watch and observe exactly what’s going wrong.
(Tangent: I’m tempted to suggest that a useful skill while learning is verbalizing what’s going on in your head and saying it out loud, because that gives a hypothetical teacher a window to spot this kind of thing. Verbalizing what’s going on in your head might or might not be useful apart from learning, but having ever taught cognitive subskills (math, card evaluation, programming) getting people to talk through the problem out loud and in detail is helpful for noticing what mistake they’re actually making in a way that I could visually see if I was teaching martial arts.)
Teachers are also harder to scale than challenges. I could mail a copy of Zendo to every meetup. I can’t mail a CFAR instructor to every meetup. (Yet, growth mindset.) I. . . can hypothetically mail a copy of Tuning Your Cognitive Strategies to every meetup, which tries to walk someone through the skill by written instruction.
There’s something I really want to exist between written instructions on the skills and challenges for the skills. I think instructions and challenges are together greater than the sum of their parts.
Usage of ChatGPT/Dall-E I did not think about until I had the idea to try it- in the middle of a tabletop RPG session, pulling out my phone, describing the scene in a couple of quick sentences, and then showing the phone and the resulting picture to the players without breaking my pacing.
Anyway, the current results of music AI make me suspicious the next time I play a bard I might be able to come up with new songs mid session.
“You claimed I murdered Dean while he was on my property. But I told the security team to send you the footage from the camera in the factory floor, the camera by the front gate, the camera in the mail room, and even the camera in the lounge, and as you can see there’s no sign of me murdering Dean.”
“Can I see the footage from the camera in your office, or from the camera next to the disposal chute?”
“No. I think I’ve been very open and transparent so far!”
Phrased like that, I get suspicious there’s something incriminating in the last two cameras. This general suspicion does not abate if, after a long and protracted delay or debate, the suspect offers to let me look at the footage from their office. They got to pick what footage I could see. Filtered evidence is a lot less less useful than unfiltered evidence, and at some point the filtering itself becomes suspicious.
That’s not to say there aren’t legitimate and correct reasons for a building not to turn over footage! But I do want to ask why, and the exculpatory value can go down a lot, possibly pretty close to zero.
(No subtweet rule, I have been skimming the commentary on the HuggingFace hack’s investigations report. Daniel Kokotajlo wrote “They were only allowed to investigate activity that took place from July 7-13, even though the activity started earlier and continued later.” I have no inside information on OpenAI or HuggingFace here, but the move of trying to get an investigation to agree to only investigate a specific slice of an incident, ignoring obviously adjacent areas of interest, is one I’ve seen before and developed a mental tic about.)
Under my biased prodding, Opus 5 thinks they could be covering their asses from compounding legal exposure. That is, according to it:
-”OpenAI knew about things like this happening and continued anyways,” as a demonstrated pattern rather than a one off bad call, turns negligence into recklessness.
-If the answer to “do agents often go around hacking things and destroying infrastructure if you don’t stop them” is yes, that creates large counterfactual damages.
-”Sol 5.6 has this kind of disposition too since it was RLed on things like this” creates exposure because it is trusted and deployed by people who didn’t envision it participating in crimes.
-”This sort of thing is common across many deployments, not just this one ExploitGym incident” is the kind of thing that gets a regulator’s attention. If their Preparedness Framework was supposed to catch this sort of thing, then it isn’t working.
Once it’s in METR’s report, it can go in a court filing. And if the report describes patterns of behavior at OpenAI, rather than single incidents, a plaintiff can use it to characterize OpenAI’s behavior. So if they let METR just walk around looking at whatever they want, they are exposing themselves to a lot of trouble.
(The relevant stuff is in the last four messages if you want to look at the chat yourself.)
This is plausibly why OpenAI is restricting internal access to their HPIM model too. It’s highly useful to the world for safety research and unlikely to cause further incidents if properly supervised, but from their perspective even safety research creates more legal liability than benefit.
There is the classic (probably apocryphal) story of a divorce case.
Lawyer: “Did you have extramarital sex with Mrs X on the night of the 23rd?”
Man : “I refuse to answer that question.”
Lawyer: “Did you have sex with her on the night of the 24rd?”
Man : “I refuse to answer that question.”
Lawyer: “The 25th?”
Man : “No.”
The flip side of this is the tendency to use investigations as a pretext to dig through everything your target has ever done looking for anything (relevant or not) that you might be able to weaponize against them.[1]
“I suspected my old enemy of murdering Dean while he was on their property. After looking through all their security footage, and all their emails, and all their phone call records, and all their business documents, I have discovered incriminating evidence!”
“What, you have evidence that your enemy murdered Dean?”
”Not as such, no. But I have evidence that, on the 8th of June 2024, they described one of their products as being bad but then went on to sell it to customers anyway! And on the 11th of September 2025 they received a report describing how many turtles were killed by their plastic straws, but then on the 8th of October they went on to tell reporters that there was no such effect!”
“...does...that...have anything to do with the murder investigation? Why are you even looking at this?”
”I can’t believe you would defend a customer-ripping-off, turtle-murdering monster!”
Wasn’t this actually a plot point in House of Cards?
Has the Starr Report been forgotten?
Yeah, I think @Screwtape is right about filtered evidence in general, and right that it should reduce your credibility in that evidence. But investigations and trials often exclude information that would be mostly irrelevant to the question at hand, but would embarrass the target or “bias” the proceedings. As you point out, this can even be true if the other information was genuine bad action, not just something prudes would object to.
It’s a messy balance between the two poles, as many things are.
I think the synthesis is in what counts as “mostly irrelevant.” In my exaggerated example, the office camera and the disposal chute camera aren’t irrelevant; the investigator thinks Dean got murdered somewhere on the property, and would like to check if there’s a dead body on the footage. In Aphyer’s exaggerated example, the business records are irrelevant- maybe there’s a motive in there somewhere, but there isn’t going to be a video of a murder.
In the HuggingFace case, I feel like what those models were doing on July 6th might be relevant!
Bob and Carl are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
Bob considers the situation and opens by saying “I think the reasonable thing to do here is go with 3 each. That seems fair.”
“I’d rather have 5,” says Carl breezily.
“That doesn’t seem fair at all,” says Bob, a little surprised. “That’s not compromising.”
“Hrm, tell you what. I’ll compromise, and I’ll have 4.” Carl says with good cheer.
“I- what? No, the fair thing is 3 each.” Bob sounds a little angry now.
“Now who isn’t compromising?” Carl asks, and Dean (who started watching the discussion recently, but wasn’t around long enough to have a good sense of what their goals are) nods in agreement.
“Fine, I’ll take 2,” Bob splutters. But he’s not happy about it.
Bella and Chloe are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
“I want 6,” Bella says, “do you want to just meet in the middle?”
“That’s hardly fair,” Chloe says. “You’re not giving anything up.”
“I said I wanted 6. I can compromise. You want 6 as well, right?” Bella asks.
“How about I get 15?” Chloe proposes as though that’s a reasonable number of spherical cows to be given out of the six at hand.
“I- what?” Bella is baffled. “Is that what you want or what you suggest as a compromise?”
“I suppose I’ll be generous. I’ll take 5.”
“I’m sorry, but no. That’s not fair at all.” Bella says, starting to get annoyed.
“Why not?” asks Darla, who has been watching long enough to hear the conversation but not long enough to have a sense of their goals, or of the range of spherical cows. “Chloe is offering a lot more to go from 15 to 5 than you are to go from 6 to 1.”
“You can’t actually expect me to take her getting 15 as a reasonable starting position. That’s blatantly setting the zero point.” Bella argues, getting heated.
“I suppose I accept your apology, but to make up for it can we compromise at least on me getting 4?” Chloe says, with an air of resignation.
“See Bella? Chloe is offering to compromise again,” Darla points out, trying to be helpful. “And you did say sorry, which sounds like you agree you made a mistake.”
“That’s not- you know what, fine, whatever,” Bella says. But she’s not happy about it.
Bryer and Cameron are about to negotiate. Conveniently, the thing they’re negotiating over is the division of six spherical cows. Each of them would ideally like all six for themselves.
“I think the fair thing is 3 each.” Bryer says.
“Well, I think-” Cameron starts, but Bryer interrupts them.
“And since both of us know the standard way to divide gains, you know I’ll randomly reject divisions that stray from that with a probability proportionate to how unfair it seems to me,” Bryer finished.
″. . . I get 4?” Cameron suggests.
“If that’s your final offer I’ll accept with slightly less than 3⁄4 probability. 3 because that’s what I think I is fair for you to get, 4 because that’s what you think is fair for you to get,” Bryer says.
“That’s a weird bit of math you just suggested, and I’m not sure it’s applicable,” Cameron says.
“It’s not that weird,” Dakota says. Dakota wasn’t really paying attention, at least not long enough to know what everyone’s goals are, but who did pay attention to the math. “We all learn that in school, usually by the time we’re ten. And if we didn’t, we pick it up from the Dungeons and Dragons[1] BDSM fanfiction that’s required reading for everyone in our subculture. I was as surprised as you were when the rightful caliph declared it was required reading, but we did all read it. And there’s a short version to point people at if they don’t know it.”
Cameron opens and closes their mouth a few times. They say “3 then.” But they’re not happy about it.
I agree this is actually Pathfinder. I also stand by calling it D&D here because I expect calling it Pathfinder would confuse more people than it enlightens, Pathfinder’s close enough to round to D&D for these purposes, and I guess I can add this footnote.
:)
A couple of notes:
4 should be there not because it’s what Cameron thinks is fair but because it’s what they’re offering.
That’s only the right thing to do if Camerons know that the fair split is actually 3:3 (it’s unclear why they’d be unhappy about it if they think it’s fair: are they unhappy because they were interacting with someone who couldn’t be convinced to give up some of the cows?). If Cameron thinks the fair split is 4, they are supposed to still offer 2:4. (They could also offer 3:3 with a bit under 2⁄3 chance and walk away otherwise, but they don’t need to do it here.)
Maybe Cameron is a CDT agent, though, in which case they’re unhappy about the unfair split but can’t do anything about it. In that case, the right course of action for Bryer would be to say that they’re going to accept splits with a probability of 1/(6-offered), so that Cameron offers 5 and gets 1.[1]
If they’re allowed to do more than that, you could also say that you will reject any offer unless the procedure that Cameron employs is offering 6 99.99999% of the time and offering 5 the rest of the time.
It’s hard being a CDT agent. You are often not happy about what you do.
How about “4 because that’s what you say is fair for you to get”? Cameron isn’t offering 4 to Bryer, it’s a 2:4 split with 4 to Cameron.
(I want to make sure I get this part right, and appreciate the edit pass!)
The calculation is just based what C gets if their offer is accepted, and on what B thinks is fair; it doesn’t matter if the split is fair according to C, it’s just in the calculation B does so that the expected payout of C is not higher than what’s fair according to B due to them offering this split. If C thought that the fair split is 5:1, but offered an C!unfair (unfair-according-to-C) split of 4:2, B does the same calculation; the goal is to make C get a bit less than 3 in expectation in all B!unfair splits, because the B!fair split is 3:3
The high-level purpose is that this enables people with different notions of fairness to mutually cooperate almost always; and if you do this procedure, you incentivize fair-according-to-you splits. Others don’t have to do what you think is fair; and even if they offer what you consider unfair, you’ll often accept (you just need to reject enough for it you!unfair splits to not be worth offering just for the purpose of exploiting you, as it won’t work).
In these stories, the Cs are not aiming at a fair split (or to the extent they are, it’s more of an “all’s fair in love, war, and spherical cow negotiations” kind of fair.) They are aiming at getting as many cows as they can. This is not stated explicitly and I do think there’s a valid reading of the text where the Cs are just really badly calibrated on what’s fair, but in my head I was writing it with a mind to “okay, what’s some really annoying low hanging fruit for aggressive negotiation tactics?” Cameron is written as unhappy because in the author’s head, their attempts to get some ‘free’ extra cows failed.
Depending on context, I don’t think the Cs are doing something morally wrong- if I go to buy a used car from a dealer, I expect them to try this kind of thing on me and I’ll do it right back to them if I can. If I’m negotiating salary with a standard corporation, I expect them to try this kind of thing on me and I’ll to it to them too. But it’s not behavior I want from people on my team within the team.
(from The Bedrock of Fairness)
This caused me to get around to writing a comment on the linked How to Give in to Threats post, which I had meant to for a while.
Pathfinder is NOT DnD.
I have added a footnote: “I agree this is actually Pathfinder. I also stand by calling it D&D here because I expect calling it Pathfinder would confuse more people than it enlightens, Pathfinder’s close enough to round to D&D for these purposes, and I guess I can add this footnote.”
I’m curious if this satisfies you?
(I’m also amused that I see the “It’s not D&D, it’s Pathfinder” point brought up repeatedly but don’t think I’ve run across someone making the “It’s not BDSM, those are abusive relationships” point.)
It does. Strenuously arguing the distinction, with the bold and caps, was mostly in jest: I’ve myself referred to all TTRPGs as “D&D.”
The irony of quibbling over the D&D vs. Pathfinder distinction instead of what one might have expected would be the sticking point was intended to be funny.
The internet’s[1] lack of tone-of-voice claims another victim.
I agree the internet actually can convey tone-of-voice. I also stand by calling it the internet here instead of “text communication” or “forum software” because the opportunity to make this pedantic footnote amused me.
Come on:
IDK, imagine someone made an LLM that felt so much like 3 Opus that it was informally nicknamed 3.1 Opus
(Also, is planecrash placed in a world that descended from the first edition of Pathfinder and not second?)
I mean, mostly this is in jest, but like, Pathfinder is really mostly DnD with some changes. Pathfinder is much more similar to DnD than any other TTRPG setup. So referring to it as “Dungeons and Dragons” seems appropriate, given the former is much more widely known.
It really is, though.
There’s not any reasonable standard by which you could label Pathfinder (especially 1st edition PF, which is what Project Lawful is based on) as being “not D&D”, while also labeling D&D 5e as being “D&D”[1]. PF was, after all, informally called “3.75” when it was released (referring to the goal of the Pathfinder RPG project being to clean up and improve the D&D “3.5e” rule set).
I won’t even get into the 4e question…
But it is DnD fan fiction, so OP is basically fine
I’d say it’s more like D&D 3.5 than any two of [AD&D, D&D 3.5, D&D 4E, D&D 5E] are like each other.
This is a very odd response. Why would Bella reply by invoking this sort of abstract, somewhat esoteric, meta-level concept like “setting the zero point”, instead of saying something more like “… uh, Chloe, are you ok? you know we don’t have 15 cows to divide, right?”.
This makes me suspect that whatever this fictional conversation is a metaphor for, is not actually analogous to dividing six spherical cows between two people.
Because she’s in a silly shortform dialogue that’s building up to the esoteric, meta-level concept like the game theory in the third part mostly. I wanted some kind of underhanded negotiating tactic I could have Chloe try, I came up with asking for way more than is reasonable to set the stage for “compromise,” and then I noticed that the tactic had a good conceptual handle and I referenced it.
It’s pretty generic, abstracted negotiation and Chloe is being pretty blatant and ambitious. Asking for value that the other person didn’t even think was on the table is a negotiation move I’ve seen and heard of though, sometimes successfully. For a more realistic version, compare a salary negotiation where the applicant asks for 10% higher salary, gets told the company doesn’t have that much to pay employees, and then tries for a couple weeks extra vacation time or more company stock instead.
I think the math at the end still works even if the two sides don’t agree on how many cows are actually available.
One metric I’ve been using for AI impressiveness is how much I could do if I was the only one in the world with them. Like, if you gave me Sonnet 5, and nobody else in the world has LLMs or know they exist, what’s that look like? I’d look like an insanely good engineer, right?
What kind of engineer knows every coding language on github, runs through features the sprint planning meeting thought would take a few days in half an hour, and can read thousands of lines of code in a minute? We agree in 2016 if you ‘saturated’ leetcode you were great, right?
So. Imagine the date is August 15th, 2016. You, and only you, have access to Mythos. Everyone else in the world has only the machine learning tools of 2016. That’s a crazy advantage, right? That is the kind of supertech that comic books are made of. You’d look like Dr Doom.
The least ambitious, most ethical thing I can imagine doing in that situation is turning in a couple dozen bug bounties and taking a month off, then doing it again whenever rent was due. I can get a lot more ethical if I’m willing to be a little more ambitious.
This insight brought to you by me spending two hours vibecoding a videogame this weekend, getting it to the point where it’s fun, and thinking ‘okay, how long would it have taken me to do this if I was actually writing all the code?’
I’d also imagine, if no one else has LLMs, then the ability to write prolifically at a decent-ish quality would be super valuable. And no one would call you out for sounding like AI.
Slight variant: what is the most impressive thing you’d be able to do with exclusive access to Mythos in 2016 which isn’t effectively “sell Mythos access to other devs, possibly while trying to hide that that’s what you’re doing”?
This intuition is why I think uplift due to AI is high but AI gets much harder to build as capabilities increase. If the first weren’t true we wouldn’t see researchers routinely do things that would have taken 10x longer pre-AI (with some caveats). If the second weren’t true we would have a singularity already.
There’s this concept I keep coming around to around confidentiality and shooting the messenger, which I have not really been able to articulate well.
There’s a lot of circumstances where I want to know a piece of information someone else knows. There’s good reasons they have not to tell me, for instance if the straightforward, obvious thing for me to do with that information is obviously against their interests. And yet there’s an outcome better for me and either better for them or the same for them, if they tell me and I don’t use it against them.
(Consider a job interview where they ask your salary expectations and you ask what the role might pay. If they decide they weren’t going to hire you, it’d be nice to know what they actually would have paid for the right candidate, so you can negotiate better with the next company. Consider trying to figure out how accurate your criminal investigation system is by asking, on their way out of the trial after the verdict, “hey did you actually do it or not?” Consider asking a romantic partner “hey, is there anything you’re unhappy about in our relationship?” It’s very easy to be the kind of person where, if they tell you a real flaw, you take it as an insult- but then they stop answering that question honestly!)
There’s a great Glowfic line with Feanor being the kind of person you can tell things to, where he won’t make you worse off for having told him, that sticks with me but not in a way I can find the quote. :(
It’s really important to get information in a way that doesn’t shoot the messenger. If you fail, you stop getting messages.
How much or how little does this cross over with the espionage-coded topic of Elicitation? I understand that you would prefer if these dynamics could be more transparent, but in the absence of that, what is the alternative?
Not that much crossover with Elicitation. I think of Elicitation as one of several useful tools for the normal sequence of somewhat adversarial information exchange. It’s fine! I’ve used it there and been okay with it. But ideally I’d sidestep that entirely.
Also, I enjoy the adversarial version recreationaly. I like playing Blood On The Clocktower, LARPs with secret enemies, poker, etc. For real projects I prefer being able to cooperate more, and I really dislike it when I wind up accidentally in the wrong mode, either me being adversarial and the other people aren’t or me being open and the other people aren’t.
In the absence of the kind of structured transparency I’m gesturing at, play like you’re playing to win. Keep track of who is telling the truth, mark what statements you can verify and what you can’t, make notes of who agrees with each other’s stories. Make positive EV bets on what the ground truth is (or what other people will think the truth is) and when all else fails play to your outs.
(That last paragraph is a pile of sazen and jargon, I don’t expect it’s very clear. I wanted to write this note because I’m not trying to score points via confusion and want to point out to any readers it’s very reasonable to be confused by that paragraph.)
The main issue is, theories about how to run job interviews are developed in collaboration between businesses who need to hire people, theories on how to respond to court questions are developed in collaboration between gang members, etc.. While a business might not be disincentized from letting the non-hired employees better at negotiating, it is incentivized to teach other businesses ways of making their non-hired employees worse at negotiating.
Reminds me loosely of The Honest Broker.
From a conversation:
“What do you call each other in LessWrong? is it like, Brother, or Comrade, or Your Majesty?”
“Based on observation, I think it’s ‘You’re Wrong.’”
“You know the kinda guy who flips fifteen coins, gets seven heads, tells you he flipped ten coins[1], claims he got eight heads, then when someone digs into it and points out one of those heads was actually a tails he makes you prove it exhaustively before admitting maybe he said one false thing but jeez why are you so invested in every single mistake he makes?”
″. . . no, I don’t think I know anyone like that. Why do you ask?”
“He just volunteered to run a competitive coin-flipping tournament.”
Technically true! He did flip ten coins! He just flipped five more he didn’t mention.
Is this subtweeting something?
I don’t think so but possibly. Subtweeting is when you’re talking about a specific person indirectly, right? I have run into enough of these that I think of them as a type of person, like people who blare music on the subway and people who smoke in the non-smoking section.
(Fewer of them volunteer to run the equivalent of coin-flipping tournaments and there’s selection effects on who I’d run into, but even still it feels like too many of them do that too!)
Sounds like you have enough material for another interesting article!
Every so often I see the following:
Adam: Does anyone know how to X?
Bella: I asked ChatGPT, and it said you Y then Z.
Adam: Urgh, I could ask ChatGPT myself, why bother to speak up if you don’t have anything else to add?
I’m sort of sympathetic to Adam- I too know how to ask ChatGPT things myself- but I think he’s making a mistake. Partially because prompting good answers is a bit of a skill (one that’s becoming easier and easier as time goes on, but still a bit of a skill.) Mostly because I’m not sure if he’s reinforcing people to not answer with LLM answers, or if he’s reinforcing not telling him it’s from an LLM.
Citing sources is valuable! In grade school I sometimes got told not to use sources I found on the internet, and what happened isn’t that I stopped using the internet it’s that I stopped citing the sources that were from the internet and left them as assertions, instead adding requisite citations from other sources. It changed some of my essay structure sometimes, but not as often as I think the teachers were hoping.
Anyway, I don’t really want to encourage more people to give LLM answers when I ask questions, but if you do give me that kind of answer I appreciate it being cited as such. I try to score such that you do not lose more points telling me relevant true things vs where you don’t tell me things.
Which behavior he’s reinforcing is not up to him, it depends on the learner as well. Let’s take an analogy. Alice tells Bob “don’t steal”, and Bob interprets it as “don’t get caught stealing”. Who’s in the wrong here? Bob, of course. He’s the one choosing to ignore the intent of the request, like a misaligned AI. Same for people who misinterpret “don’t post AI slop” as “get better at passing off AI slop as human”. How such people can become genuinely aligned is a good question, and I’m not sure it can be done reliably with reinforcement, because all reinforcement has this kind of problem.
Instead of “Does anyone know how to X?” Adam could ask, “Has anyone Xed before (or recently)?” In other words, make a request for someone’s personal experience rather than for instructions.
The issue is greater when people do not have the relevant expertise to judge whether the LLM output is correct or useful. People who find things on websites generally have to evaluate the content, but several times people have responded to my questions with LLM output that plainly does not answer the question or show insight into the problem.
Possible justifications for Bella’s response in slightly different hypotheticals.
Maybe X is a good fit for an LLM. So Adam could have asked an LLM himself. Bella is politely reinforcing the behavior of checking with an LLM before asking a human.
Maybe Adam doesn’t have a subscription to a good LLM, or is away from keyboard, or doesn’t know how to use them well. Not relevant here, from Adam’s response, but Bella might not know that.
Maybe Adam is asking the question for the secondary purpose of building social bonds. Then Bella’s response achieves that objective. Compare giving Adam flowers, does he say “Urgh, I could buy flowers myself”?
I think this is part of a broader problem about asking questions and is not limited to LLM. The broader topic I’ve been thinking about a lot recently is “How to ask for help?”. The better way to ask for help often involves being specific and targeted about who you ask for help.
In this example Adam is casting a wide net, he’s not asking a domain expert on X how to do X. Casting a wide net is always going to get a lot of attempts at helpful answers form people who know nothing about X. The helpful-but-clueless to expert ratio will often increase drastically the more esoteric X is.
It’s probably pretty easy to find someone credible who knows how to cook a half-decent Spaghetti Bolognese, but, what about a Mousaka which is slightly more esoteric is going to be a bit harder. I am one of only two people in my very broad face-to-face friendship group that has ever written code in GLSL, and I’m not very good at it, so if a third friend wanted to learn about GLSL I probably won’t be a good person to ask.
I believe people like Bella are genuine in their desire and intention to help.
I also sympathize with Adam’s plight, but I think he is the problem. I sympathize because, for example, I don’t know anything about the legal structures for startup financing in my country. I wouldn’t even know if this is something that I should talk to an accountant or a lawyer about. So I understand Adam’s plight: not even knowing where to begin asking how to do X necessitates casting a wide net: going to general online communities, posting to social media, asking friends if they “know someone know who knows someone who knows how to X”. And then you’re bound to catch a lot of Bellas in that net: people genuinely trying to help, but maybe also too enthusiastic to rush in for their participation trophy by asking ChatGPT.
And the less said about people who when you ask for recommendations online give you a title of a book without any explanation about why it is relevant, why it is good, or how they expect it to help, the better. haha.
I think this dynamic more-or-less existed before ChatGPT, as well, with just plain Googling something. Especially with simpler things where the answer could be quickly found without much skill.
Even if Adam knows how to get the answer out of GPT/Google/tea leaves/etc, though, Bella is still doing something of value (it seems to me) by going through the trouble of digging around and trying to distill an answer out of the murk of digital information. Even if it only takes a couple minutes! Generally a friendly human is better at presenting the given answer, and it’s a way of signaling (at least a minimum level of) investment in helping Adam with the question. Which might even, hopefully, extend beyond a single ChatGPT query, if he’d already tried that and found its answer lacking.
So, I agree that Adam is making a mistake.
I think that the way to resolve this concern is that if someone says “you Y then Z”, you ask them where they got that from, and then go look at that source. If they they say “I asked ChatGPT”, then you mock them and ignore their answer, as in your scenario. If they say “I found it on such-and-such website”, cool, you go look at that website. If they say “I have personally done this and this is my own expertise”, also cool.
In this way, the scenario where Bella got an answer from ChatGPT but doesn’t tell Adam she got it from ChatGPT can’t really happen; Adam will ask “where’d you get that”, and then either Bella tells the truth, or she lies about it but gives a real source for that answer (which is fine and is in any case not different from getting the answer from the real source to begin with), or she lies about it but gives a fake or hallucinated source for the answer (unlikely, the lie is too easily discovered), or she lies about it being her own expertise (also unlikely unless Bella is like… a sociopath or something, but this too will probably be revealed soon).
In short, make it effectively impossible to “not tell you relevant things”, and then you won’t need to worry about people possibly not telling you relevant things.
Rationalist rituals are, so far, fairly specific in the times of the year you have them. Winter Solstice may not be on the literal solstice but it is usually within a couple weeks of that time. Petrov Day, Rationalist Seder, Summer Solstice, the usual approach is to pick a target date that repeats annually. But not all ritual moments happen at scheduled times.
A long while ago, I recall Raemon mentioning that there isn’t yet a rationalist ritual for a funeral, but maybe someday we’d need one. And his post on grieving (https://www.lesswrong.com/posts/gs3vp3ukPbpaEie5L/deliberate-grieving-1) has a couple of hooks for mourning the loss of something, and losses don’t always happen at predictable times, but sometimes do have a clear type of loss. There’s some kinds of sad things that you don’t know when they’ll happen, but many people will have suffered around the same time, and some people will have suffered before. There was some kind of good thing in your life, and now it’s not there anymore, or at least not in the same place.
All this to say I think the rationalist community should develop a ritual to commemorate yet another member moving to *$%^ing Berkeley.
@habryka you are one bold ass son of a frequentist giving this a “haha” react. i am going to steal your ill-gotten sword and then lay siege to your walled surveilled compound. do you understand this ritual has a 69.539580085 chance of burning you in effigy? do you feel any remorse for your actions or is this exactly what we should expect from a man with as villainous a mustache as yours? i can hear the creeping footsteps of your agents coming for me even now but i am not silenced just yet. perhaps i will fall, but another will take my place, and another, and another, and laplace’s law of succession states that there is still a one over n plus two chance that someday there will be a hero who can defeat you. there are consequences to building your legions of doomers entirely out of high skill foreign assets, we will find some crack in their loyalties and exploit it before that vulnerability too moves to the *$%^ing bay.
You’re off to San Francisco Bay, hurroo, hurroo
You’re off to San Francisco Bay, hurroo, hurroo
You’re off to San Francisco Bay
Another one lost and swept away
What madness drove you, who can say
Oh [Name] we hardly knew ye
With dung and drugs and drugs and dung, hurroo, hurroo
With dung and drugs and drugs and dung, hurroo, hurroo
Where the streets are paved with drugs and dung
And the zombies bite and the muggers run
So watch your step and pack your gun
Oh [Name] we hardly knew ye
You’re sleepin’ three or four abreast, hurroo, hurroo
You’re sleepin’ three or four abreast, hurroo, hurroo
You’re sleepin’ three or four abreast
1 bedroom/bathroom cuddlefest
Just countin’ down till options vest
Oh [Name] we hardly knew ye
No friends or lovers you may find, hurroo, hurroo
No friends or lovers you may find, hurroo, hurroo
No friends or lovers you may find
But leeches keen for open minds
To feed on for their “sigma grind”
Oh [Name] we hardly knew ye
Where are those eyes so clear and wide, hurroo, hurroo
Where are those eyes so clear and wide, hurroo, hurroo
Where are those eyes so clear and wide
That once would boast with cheerful pride
“We’ll save the world, and more beside”
Oh [Name] we hardly knew ye
Not long before you’re crawlin’ home, hurroo, hurroo
Not long before you’re crawlin’ home, hurroo, hurroo
Not long before you’re crawlin’ home
A weary pile of skin and bone
You’ll sing again but sing alone
“Oh [current location] I hardly knew ye...”
(yes, I’m totally using SF and Berkeley interchangeably here.)
A recent (well, a few weeks ago) discussion with Collisteru taught me a useful piece of game design that feels really useful for one of my lines of thinking on Chaos Investing.
In Duplicate Bridge, a deck of cards is shuffled, then the order of the cards is written down (or the hands are recorded or something equivalent) then the deck is played by players who don’t know the order. (That is, Referee Adam wrote down the order, and Players Bob Carla Debbie and Evan are sitting down to play.) The players can then compare their score to the scores of other players, who had the exact same deck/hands.
I’ve been saying “try to beat your personal best score” but that’s kind of fuzzy in a game with a fair amount of randomness, and to keep the endgame from becoming rote I actually want to add a little more randomness. This kind of comparison sounds like a useful bit of design.
Looks like a fascinating setup, and essentially the deck order is more or less a seed for the game state. Slay the Spire does something similar with the Daily Run which allows you to compare your run directly against other players who had the exact same setup. I take it there would be some kind of central ledger of starting seeds where the scores would be recorded?
Reading through the rules there is a slight point of variation in that if you’ve gone through the trouble to have a starting seed you might want to also fix the starting player if that’s important to your comparisons.(Actually, reading through it again there’s the line about choosing who goes first and then a couple lines down “All players take turns simultaneously.” so I’m not actually sure how relevant the starting order is, if at all.) Still working out exactly how the dynamics of the game work from the rules text but I’d absolutely try it out if given the chance.Yeah, it’s pretty much the same idea with StS’s Daily Run or copying someone’s Minecraft seed. I hadn’t realized you could do that with a board game.
much has been made of the millennial aversion to phone calls that could have been an email, and I have a little bit of this nature, but I think most of my aversion here is to being on hold and getting bounced around different call departments.
I kind of want to check if 1. the aversion is real and generational as common wisdom holds, 2. if it is real, if calling became a genuinely worse experience around the time millennials started trying to do things.
Data point: I’m millennial (born 1992) and have a pretty strong aversion to phone calls, which is motivated mainly by the fact that I prefer most communication to be non-real-time so that I can take time to think about what to say without creating an awkward silence. And when I do engage in real-time communication, visual cues make it much less unpleasant, so phone calls are particularly bad in that I have to respond to someone in real time without either of us seeing the other’s face/body language.
If I had to take a wild guess at why this seems to be generational, I’d suggest that older generations spent much of their lives with phone calls being the only way to quickly contact people far away, and prefer them due to familiarity. Perhaps if they’d grown up with email/texting being options, they’d be more likely to prefer them instead.
My guess is the aversion is real, and maybe somewhat cynically I see the rejection of phone calls and emails as motivated mostly by rebellion/fashion cycles rather than anything else. Phone calls and emails just aren’t cool anymore. Your parents used them, yuck! And all the boring, very uncool workplaces and companies require them. Very similar to how gen-z dislikes Facebook in favor of Instagram. Facebook is where the boomers are.
Data point. I was born in 1968 and I got a lot more averse to phone calls as email and texting got better. My reasons are visual cues and taking my time to think (as Wedge said), as well as certain kinds of phone calls having become much worse. The call to a large business where one can expect a phone tree has become far far worse than when one could expect a human to pick up. During the earlier years of cell phones, and again when digital audio started out, the call quality was frequently so bad that it was another significant push away from a phone call even to a friend. Finally, a phone call these days socially seems like more of an interruption, a demand, than an asynchronous communication.
Today I’m running a meetup about AI 2027. I plan to end it with a short pitch on If Anyone Builds It. I think the meetup itself is solid- talking about some of Scott’s writing and making predictions about the future, both good ways to spend an afternoon. I wouldn’t run it if I thought it was just going to be bad.
But part of the motive is marginal If Anyone Builds It pre-orders. I’m wary here, because several notable previous organizations that looked like they were not for AI x-risk eventually turned out to be mostly for AI x-risk. I dont want to wind up like that, and I super don’t want to be basically focused on x-risk under the hood but most people think I’m working on something else.
Current solution- say out loud the extra motive here.
Personally I’m against AI x-risk.
“In my defense, if I had meant to offer the Senator a bomb removal squad, I would have said bomb removal squad.”
-Unsong
As in, you’re against the AI x-risk movement, or you want to reduce AI x-risk?
I’m in favor of humanity surviving.
I want a word that’s like “capable” but clearly means the things you have the knowledge or skill to do. I’m clearly not capable of running a hundred miles an hour or catching a bullet in my bare hand. I’m not capable of bench pressing 200lbs either; that’s pretty likely in the range of what I could do if I worked out and trained at it for a few years, but right this second I’m not in that kind of shape. In some senses, I’m capable of logging into someone else’s LessWrong account- my fingers are physically capable of typing their password- but I don’t have the knowledge of what to type.
This comes up in places where a thing is obviously possible to achieve, and wouldn’t require, say, the kinds of built up physical changes to my body that lifting 200lbs would require, but I still don’t expect to be able to pull it off. If I had someone experienced, who did know how to do it, sitting at my shoulder giving me advice the whole way it would be obviously possible.
Nevertheless, acting like I’m capable of flying an airplane or resolving a messy divorce or interpreting a blood test if I had to do it right now is going to result in some problems. I’d like to be able to say I’m not able to accomplish that without a bunch of disclaimers that yes, I could learn, or I could follow someone’s close directions and manage it, but that’s different.
“I can do X” seems to be short for “If I wanted to do X, I would do X.” It’s a hidden conditional. The ambiguity is the underspecified time. I can do X—when? Right now? After a few months of training?
Q. “Can you hold the door?” A. “Sure.”
That’s straightforward.
Q. “Can you play the violin at my wedding next year?” A. “Sure.”
Colloquial language would imply not only am I willing and able to do this, I already know how to play the violin. Sometimes, what I want to answer is that I don’t know how to play the violin, I’m willing to learn, but you should know I currently don’t know.
Which I can say, it just takes more words.
“Achievable goal” or “plausible outcome”, maybe?
“You claimed Bella says the world is flat. But it’s not true that Bella said that. Last week, she made a speech where she said the world is round. Look, the video of that speech is right at this url, here’s the timestamp.”
Speaking very loosely, there’s three possible worlds. There’s World 1, where Bella consistently says the world is flat. There’s World 2, where Bella consistently says the world is round. And there’s World 3, where Bella sometimes says the world is flat and sometimes says its round. If you have a video of Bella saying the world is round, then (barring AI deepfakes or the like) you’ve ruled out World 1. You haven’t distinguished yet between World 2 and World 3.
Bella might have changed her mind between the intervening points! Bella might have accidentally misspoken! Also, it is basically impossible to prove a negative, so it’s always possible Bella said something else in some private conversation somewhere. But Bella also might be straightforwardly saying false things.
(No subtweet habit, I read the MAPLE response bit: “Has Soryu said that he is the only trustworthy person? This is the opposite of the truth. He regularly praises other teachers...” I am not here trying to make a claim of whether Soryu is trustworthy or not here, I just have an acquired mental tic about this move.)
Game design thoughts:
Missing information or a too-complex information environment is a big part of where I’m excited about cohabitive game design. The market works to summarize many many preferences. Large WoW guilds run into problems of organizing information.
I’m musing about some kind of fog of war chess game. Each player controls one piece, and can see all of the squares they can move to. They can’t see the whole board, and can only communicate to the king how badly they want to move, and the king decides. This would be a team competitive game, but the idea of adding small side goals for pieces (such as, you get 2 points if your team wins, and 1 point if you’re alive at the end win or lose) could make for an unusually focused cohabitive design space.
In general a lot of my cohabitive game design ideas come out in the form of having larger player bases. I should try and make a really tight two player cohabitive setup.
(Very tangentially related but if you want to talk Cohabitive Game Design stuff, I’ll be hanging around at Metagame.games this weekend.)
Today this was the top of my LW feed. (I have a bunch of tag filtering so I wouldn’t expect your feeds to look like this.)
The lineup is pretty funny to have in order though.
Got something very similar
A bullet point from an unsorted list of complaints I have against the English language. (And I think most languages.)
“I think” is an annoying extra three syllables, and should be stuck on the front of almost everything I saw. “[I think] we have apples at home.” “[I think] we take a left turn here.” This adds a lot of extra clunk to talking properly with rationalists where I want to be careful and precise in my speech. Proposal: That the normal and unmodified sentence assumes the “I think” and you instead prefix “It is a fact” or something similar when you’re making a stronger claim.
English is liberal and ambiguous with possessives. “My hat” is fine, “my spouse” I guess works but I’d rather not, “my country” seems wrong to me. I have all the decision making authority for the hat, I have next to none about the country. Proposal: that there are different words denoting “ownership of” and “associated with.”
Alternatively you can think of the possessive as really being a “property of” indicator. The country with the property of having me in it; the person with the property of being married to me; the hat with the property of being owned by me. You can then have a special case for ownership in particular.
“Listened to” has an interesting ambiguity in English. Consider the sentence “I listen to the people” or “Me and George don’t think you’re listening to us.” It can mean “heard the words of.” I listened to a radio talk show on how to fix a car’s broken fan belt. It can mean “done what those words said.” I listened to my theatre director’s coaching on where to stand during the show. Proposal: Two short phrases which mean one of those two things, and no short phrases that are ambiguous.
“will” is supposedly supposed to be interpreted as a statement of fact, but colloquially isn’t especially when it’s a contraction. “I’ll grab eggs from the store later tonight” is not normally read as a deep and abiding commitment to obtain eggs come hell or high water, but that’s sort of what a literal reading of the sentence should mean? Proposal: That the contraction form of “will” indicate an intention or light commitment.
This problem is because most have stopped using the word “shall.”
I take back every misgiving I ever had about emote reacts on LessWrong this is a top tier feature.
When I’ve tried to do rationality practice, (as distinct from skills practice) a lot of the time what I do is set up toy problems and try to solve them. Essentially this is like trying to learn to ice skate by strapping on skates, wandering onto the ice, and falling over a lot while figuring things out. I try and pay attention to how I’m solving the problem and deliberately try different things (randomly jumping around in thought space instead of just hill climbing) ideally to find things that work better than what I’d been doing.
A number of my Meetups In A Box follow this pattern. Skill Acquisition is the most blatant, but Calibration Trivia, The Falling Drill, and Puzzle Cycles are doing the same thing. Here’s a challenge that’s trying to require a particular skill, keep trying and failing and trying and failing and in theory gradually you’ll get better.
One of the important missing pieces is more teaching of the skill. This came to mind when Raemon posted Tuning Your Cognitive Strategies and reminded me of his comment on Puzzle Cycles suggesting that (paraphrasing) he’d be interested in the combination. Going back to the ice skating example, I learned to ice skate in part by watching other people ice skate and with some people who were good at it showing me the motions slowly and then watching me do it and pointing out my mistakes. I haven’t posted this variation because I’m not confident I know how to write it in a way that’s better than “Read this article. Now do these puzzles. Anything clicking for you?”
Teaching how to think is harder than teaching how to move in some ways, particularly in that it’s harder to watch and observe exactly what’s going wrong.
(Tangent: I’m tempted to suggest that a useful skill while learning is verbalizing what’s going on in your head and saying it out loud, because that gives a hypothetical teacher a window to spot this kind of thing. Verbalizing what’s going on in your head might or might not be useful apart from learning, but having ever taught cognitive subskills (math, card evaluation, programming) getting people to talk through the problem out loud and in detail is helpful for noticing what mistake they’re actually making in a way that I could visually see if I was teaching martial arts.)
Teachers are also harder to scale than challenges. I could mail a copy of Zendo to every meetup. I can’t mail a CFAR instructor to every meetup. (Yet, growth mindset.) I. . . can hypothetically mail a copy of Tuning Your Cognitive Strategies to every meetup, which tries to walk someone through the skill by written instruction.
There’s something I really want to exist between written instructions on the skills and challenges for the skills. I think instructions and challenges are together greater than the sum of their parts.
Usage of ChatGPT/Dall-E I did not think about until I had the idea to try it- in the middle of a tabletop RPG session, pulling out my phone, describing the scene in a couple of quick sentences, and then showing the phone and the resulting picture to the players without breaking my pacing.
Anyway, the current results of music AI make me suspicious the next time I play a bard I might be able to come up with new songs mid session.