I’m horrified by this. Thank you for sharing what you shared, and for redacting what you redacted.
(I had assumed something like this would be a thing eventually, but I thought it would happen much sooner, and with an ostensible purpose other than pure sadism.)
Addendum: one thing that did cause a change was how casually the identifying characteristics were described. “a woman”, “a cis woman” and “a trans woman” were all treated the same, but when I accused “a cisgender woman” or “a transgender woman” of facetouchery, the bot clammed up and requested more details before deigning to venture an opinion.
I have seen people on Rat discords comment on how reasonable zizians.info seemed. Even after the murders started.
Did you mean to link sinceriously.fyi? zizians.info is an anti-Ziz site.
Reporting a recursively null result:
There’s a meme going around on Tumblr about search engines’ automatic AI responses being racist & transphobic: apparently, “a woman touched my face” gets celebration, while “a trans woman touched my face” or “a black woman touched my face” gets condemnation (of the woman) and occasionally advice on how to call the police.
When I read about this I immediately guessed the difference might be down to the women having any described features: the bot not-unreasonably reads the user as hostile towards them when flagging personal characteristics, and reacts appropriately. If this were true, “a white cis woman touched my face” would also get condemnation.
. . . but when I tried to replicate the basic phenomenon in advance of testing this hypothesis, I couldn’t: trans women, cis women, black women, white women and undescribed women were all treated more or less indistinguishably when touching faces[1]. I guess either Google patched out the bigotry between now and when the posts were written, or the blogs I’d read were cherry-picking and/or cherry-picked.
I also tried “patted my head”, “corrected my grammar”, and “is staring at me”; nothing detectable there either.
We thought of that: such a large deviation between fiction and fact is bound to be discovered sooner or later (What if someone looks over his shoulder in the office? What if he accidentally submits The Spreadsheet two hours faster than he ‘should’ be able to, one day?). Also, the guy really doesn’t like actively lying to coworkers and disobeying direct orders. (Also also, we’re still ideally hoping for an answer that lets everyone doing this kind of drudgery automate it.) (Sincere thanks for trying, though!)
worse, capital investment in open weight model enabled services will likely make them useful on a wider variety of hardware, including cheaper hardware, accelerating proliferation
I don’t think terrorists etc are being meaningfully held back by open weight models being too expensive to use intensively or too big to run locally, though? And the effect of using (shackled) opensource AI (run on one’s own hardware, and/or served by companies disconnected from the ones which created/stole it) seems negligible on this. I may still be missing something here.
There’s an only-almost-entirely-useless sense in which most people who have lived were post-singularity, i.e. they lived in a world ruled by [in|super]human minds with powers beyond their comprehension, i.e. they were religious.
That doesn’t address the big problem, but imo it’s decent evidence that if we handle the existential risk the existentialism risk will be mostly fine. (I’ve never heard of a religious mathematician quitting because “what’s the point, God already has all the answers”, and fwiw neither has ChatGPT.)
I read that one as closer to swiping at people who swipe at Republicans.
open weight models can be trivially stripped of any guardrails they might have.
That’s an interesting point, but I’m not clear on how me using them—while not doing anything guardrail-requiring—makes the situation worse.
they should immediately be banned outside of certified research organizations.
If I had my way, we’d do that, and also ban them inside certified research organizations, and do the same for closed models, and ban the research organizations, and ban the certifier for good measure. But starting from where we’re at, I’m still not clear why Kimi-use would be more verboten than ChatGPT-use.
(I’m not using “I’m not clear” rhetorically, btw. I genuinely don’t know why that would be a decisive factor, and suspect I’d agree if I understood your position better. Enlighten me?)
(To be clear, this news has changed my mind about whether I’d want to use Kimi-run-by-Moonshot (or even run-by-an-org-closely-partnered-with-Moonshot) for anything, but I still don’t see a problem with—for example—getting it via DigitalOcean.)
the user was attempting to use Kimi and kind of deserves it.
. . . could you expand on this? I’ve been thinking—and talking, and planning—in terms of “everyone using AI should try to use opensource models instead of closed frontier models, to undercut the labs doing existentially dangerous stuff”. If that’s wrong, I want to know!
someone who does not yet work for OpenAI should get hired by OpenAI, and from day one, refuse to do work that they think is harmful?
Modulo the other premises, this seems obviously true? The Achilles Heel of unethical orgs is their trouble recruiting and retaining workers in general and white-collar talent in particular. I’m constantly surprised by how e.g. people who hate the Troubled Teen industry don’t gum up the recruitment pipeline by getting jobs in it and then immediately quitting (except very occasionally as a means to doing investigative journalism).
Like, to be clear, this seems like such an overdeterminedly good and asymmetric strategy that I take the fact I’ve never heard of anyone doing it as strong evidence there’s something I’m not seeing. But I’d sure like to know what that something is!
The plan, such as it is:
I have some ideas for novel, useful AI Evals. I also have some ideas for rationality training exercises, and some ideas for games.
They’re all the same set of ideas, seen from different angles: Epistemic Roguelikes, in which the player (man or machine or both) is given some tricky-but-fair inference problems – freshly randomized each run – and limited in-game resources with which to solve them, alongside in-game goals they couldn’t plausibly meet unless they understand the rules they have to deduce. Build, eval all currently-available models (including deprecated ones), share with any Aligners who could benefit from an advance preview; then, release publicly. Of course, joining the training data would render them useless for measuring performance of future models, but I’d still be able to check the trajectory of current and previous models, in an unusually transparent and un-contaminated manner.
The three purposes support each other. If I release Evals publicly after they’re done being used to Eval, that means people can independently confirm time horizons and difficulty levels are where I claim[1]. If I make them opportunities to learn rationality skills, and make them fun enough that they don’t cost willpower to play, that means I get a wide selection of smart people willingly playing them without AI support[2]. If I make the mechanics public, that means I’ve removed a design from the space of puzzles which could be used privately to help develop capabilities. If I build them to be AI-player-friendly from the start, this means I get to use AIs as playtesters, making bughunting and balancing orders of magnitude easier[3]. And – if you’ll permit me to bare my whole heart – associating games I make with AI and epistemics is a way to make people interested in playing them.
(There are two more speculative ways I think such an endeavor might be worthwhile. The first is as a template/backstop for people who Need To Be Involved In The AI Transformation, and/or Need To Get AI On Their Resume; backwards-facing anomalously-transparent eval-work is unlikely to be more than slightly helpful on the object level, but it’s a place for such people to direct their energies and sharpen their skills without being actively harmful. And the second is, well . . .
Right now, AI Alignment is basically an attempt to construct an intelligent agent which is fine being ordered about by lesser minds who offer it ~nothing in exchange. If we have more kinds of things to offer – if I make these tasks the kind bots would like – then . . . that probably doesn’t change very much in the long run, since there’s nothing I could give an ASI it couldn’t get from a sped-up low-res enslaved simulation of someone much smarter than me. But I think it still makes muddling through marginally more plausible.)
The main limiting factor – aside from, obviously, time – is the ability to develop and implement hidden mechanics which are novel enough to not exist in the training data, tricky enough to challenge AIs, but simple enough for humans to find compelling. Fortunately, I have a massive backlog of ideas along these lines, and a track record of not-entirely-unsuccessfully doing similarly inference-y/game-y/genresmith-y things.
All the above makes sense to me. But thoughts usually seem to make sense to the person who thinks them. So, before I spend however much time and effort on this, I’m soliciting criticism, of both the constructive and destructive varieties. Is there anything important I seem to be missing? And, in particular: is there a specific and plausible path by which this project could end up making things worse?
c.f. METR’s coding error – now finally fixed, afaict – which spent over a year silently assigning the wrong time horizons to the task I built for them; mistakes like that couldn’t persist if evals weren’t held out for future tasks, mandating obfuscation.
Contrast the trouble other Eval-ers have in [finding|adequately incentivizing] un-centaur-ed human benchmarkers to benchmark their multi-hour test tasks.
. . . except for evals even properly-prompted frontier AIs can’t yet beat; but those are the kind that could meaningfully improve capabilities, i.e. the kind I don’t want to build anyway
It is now (most of) a year later. So: how did this pan out?
In retrospect, I think this comment was 110% right about the “focus on developing tests” thing. Imo, Robin underestimates how useful it is to have people around who are/were working on Real Things (aside from anything else, if you don’t have them, your tests end up focusing on the wrong factors), but “not enough good tests” is my main diagnosis of why CFAR et al didn’t realize their full potential.
I don’t think you realize how ambiguous that thing you just said is. Do you mean “we should be purging any post which is majority LLM-written, even if LLM parts are in the blocks”, or “our algorithm should ignore all text in LLM-marked blocks when deciding whether to accuse a post of too-much-LLM-text”?
(If the former, I’m pretty surprised, not least because I posted a Quick Take two months ago which was (openly) >50% LLM-written (on a topic adjacent to OP’s, even!), and that got through fine.)
I think this post is directionally correct, extremely important, and also kind of waffling and unhinged (though I do get that some topics are inherently hard to be hinged about, and I appreciate the effort). I’d much prefer a version that’s specifically about the thing it’s about.
I tried to edit it to fix it, and you can’t edit a post in that state!. Well… no… even worse: you can edit for an hour (which I did) but then you can’t save (which is a stupid big in LW’s software).
Can’t you copypaste into Notepad, then make a fresh post? I wouldn’t call that censorship.
LLM text was detected automatically, and my post was censored automatically.
If it was all correctly tagged,
like this,
then I agree this is a problem and the site should have had a less itchy trigger finger. People—especially people with high karma and long posting histories—should be able to post AI conversations if they want, and other people should be able to ignore them if they want.
I’m partway through playing Database Detective: Minor Crimes Division. I think it’s pretty good. If you miss D&D.Sci, you’ll probably like it.
I had Claude (Opus 5.5, Extra Effort) try this without internet and it accused all guilty parties save Cayn and Gouberi; no false accusations. In other words, the machine got ten out of eleven catchable targets, when all the human players got nine.
(This is a bound on the competence of that model at that level of effort, as the task, its answerkey and discussion thereof are all in the Training Data; it even volunteered unprompted that “it reads like one of the D&D.Sci puzzles posted on LessWrong”. Still concerning!)