casens

Karma: 54

casens 12 Apr 2026 14:20 UTC
1 point
0
in reply to: joanv’s comment on: joanv’s Shortform
and the DoW / Anthropic standoff started brewing then. how much did the DoW know about mythos at that time? was the demand “if you have a model this powerful, then the US military must have unrestricted access to it”?

casens’s Shortform

casens12 Apr 2026 0:30 UTC

2 points

1 comment1 min readLW link

casens 12 Apr 2026 0:30 UTC
2 points
0
on: casens’s Shortform
when the DoW / anthropic / OpenAI standoff was going down, how much did the parties know about claude mythos / glasswing? the standoff seemed so weird on a few levels: 1) how could this simple agreement/communication about “all legal use” or “domestic surveillance” or whatever result in such staggering consequences (DoW threatening to destroy anthropic)? 2) why the sudden extreme, public blowup? 3) why would the DoW resort to such extreme tactics (rather than, as was pointed out repeatedly, just having normal buisness negotiations and changing contractors)?

instead, if it was something like “you said you have a model that can exploit every OS. we need it” and anthropic said “it’s not ready / not safe / not ethical”… that explains a whole lot. but it would also imply that OpenAI said to DoW “we also have a model that’s good enough at X, Y, and Z”.

though for what this explains, it also seems unlikely that one of the most high-pressure, heavily scrutinized news events recently didn’t have anyone leaking about this for a whole month. and weird things about the drama are also explained by regular human personalities, stupidity and luck.

casens 21 Mar 2026 20:05 UTC
4 points
0
in reply to: Luke Stebbing’s comment on: Daniel Kokotajlo’s Shortform

I’m warning the wise would-be investors, including governments and the public.

you’re ascribing too much consequentialism to people who are generally not consequentialists. the mindset is more like:

AI is personally annoying to me and is obviously a scam/illusion. the only reason people are excited about it is because there are scammers flooding the zone with hype. me and my friends can’t do anything about it, but it’s at least gratifying to complain about in solidarity; the world may be insane, but i may as well point out the insanity. the wheel keeps turning and in a few years some other scam will come along.

casens 8 Mar 2026 16:59 UTC
6 points
2
on: When has forecasting been useful for you?
forecasting is so intractible-in-practice that metaculus doesn’t even use it for internal decisions, despite attempting to do so multiple times (source: i was there)

casens 25 Feb 2026 17:58 UTC
1 point
6
on: Open sourcing a browser extension that shows when people are wrong on the internet

XBOW’s own “Top 1” announcement is dated June 24, 2025--about 8 months before this LessWrong post (Feb 19, 2026), not “almost one year.”

this is borderline pedantry, and also the timing of the announcement is not significant to the thesis of the post. if your goal is to tell us “here’s what the extension is like, also be aware that some of the corrections are wrong/unhelpful” then fine, but if it’s a sales pitch you should choose a better example.

casens 15 Feb 2026 15:18 UTC
19 points
0
on: Life at the Frontlines of Demographic Collapse
this is a neat article and all, but what are the actual sources for any of this? what is the author’s relation to this story? they slide between statistics and checkable facts smoothly into anecdata of “this is the general sort of thing that happens in places like these”

casens 15 Dec 2025 21:07 UTC
−1 points
0
on: No, Americans Don’t Think Foreign Aid Is 26% of the Budget
But there’s an additional problem here. People overestimate foreign aid, and these polling methods exaggerate how much they overestimate. In the KFF data, the majority of respondents said it was below 20%, and the largest decile by far was 0-10%. KFF said that average was 26%, but their own data show that this isn’t what most people actually believe; it’s an artifact of using the wrong statistic.
the KFF data shows that the median respondent thinks foreign aid is somewhere in the 11-20% of the budget, so reporting the average is roughly 2.4 to 1.3 times an overestimate of the median belief. but reporting the average is what most people do, because most people are not very numerically literate, which is what the poll itself shows!

you’re missing the point by splitting hairs about whether the geometric average is more useful than the arithmetic mean, because the poll really does show that the public overestimates the foreign aid bugdet. the bigger issue at play is that opinion polling the public produces crazy results, like [the lizardman constant](https://slatestarcodex.com/2013/04/12/noisy-poll-results-and-reptilian-muslim-climatologists-from-mars/). or i would guess that people are not thinking that hard, and that if you offered them $10 if their answer was close to the real result, the median would become more accurate.

casens 18 Nov 2025 17:22 UTC
1 point
0
on: Knowing Whether AI Alignment Is a One-Shot Problem Is a One-Shot Problem
Jim, Garfield’s owner,
Garfield’s owner’s name is Jon Arbuckle. the cartoonist/author of Garfield is Jim Davis

casens 6 Nov 2025 23:42 UTC
3 points
2
in reply to: Josh Snider’s comment on: Legible vs. Illegible AI Safety Problems
i notice the OP didn’t actually mention examples of legible or illegible alignment problems. saying “leaders would be unlikely to deploy an unaligned AGI if they saw it had legible problem X” sounds a lot like saying “we would never let AGI onto the open internet, we can just keep it in a box”, in the era before we deployed sydney soon as it caught the twinkle of a CEO’s eye.

casens 19 Oct 2025 18:04 UTC
3 points
0
in reply to: Cleo Nardo’s comment on: strawberry calm’s Shortform
your entire analysis is broken in that you assume that an elo rating is something objective like an atomic weight or the speed of light. in reality, an elo rating is an estimation of playing strength among a particular pool of players.

the problem that elo was trying to solve was, if you have players A and B, who have both played among players C through Q, but A and B have never played each other, can you concretely say whether A is stronger than B? the genius of the system is that you can, and in fact, the comparison of 2 scores gives you a probability of whether A will beat B in a game (if i recall correctly, a difference of +200 points implies an expected score of +0.75, where 1.0 is winning, 0 is losing, and 0.5 is a draw).

the elo system does not work, however, if there are 2 pools of non-overlapping players like C through M and N through Z, and A has only played in pool 1, and B only in pool 2. i’m fairly certain you could construct a series ~200 of exploitable chess bots, where A always beats B, B always beats C, etc, getting elo rankings almost arbitrarily high.

so a major problem with your analysis was that you cited Random as having an elo of 477, and indexed your other answers based on that, when actually, that bot had an elo of 477 against other terrible (humorous) bots. if you put Random into FIDE tournaments, i expect its elo would be much lower.

casens 7 Oct 2025 18:55 UTC
10 points
9
in reply to: lc’s comment on: abramdemski’s Shortform
i think you’re mis-applying the moral of this comic. the intended reading IMO is “a person believes misinformation, and perhaps they even go around spreading the misinformation to others. when they’ve been credibly corrected, instead of scrutinizing their whole ideology, they go ‘yeah but something like it is probably true enough’.” OP doesn’t point to any names or say “this is definitely happening”, they’re speculating about a scenario which may have already happened or may happen soon, and what we should do about it.

casens 22 Jun 2025 16:09 UTC
3 points
0
in reply to: jessicata’s comment on: Consider chilling out in 2028

Though, notably, Metaculus lists Jan 2027 as a “community prediction” of “weakly general AI”. Sure, someone could argue that weakly general AI doesn’t imply human-level AGI soon after

it does imply that, but i’m somewhat loathe to mention this at all, because i think the predictive quality you get from one question to another varies astronomically, and this is not something the casual reader will be able to glean

casens 24 Mar 2025 22:43 UTC
7 points
4
on: Will Jesus Christ return in an election year?
The True Believers hypothesis rings false because that would be a frankly ridiculous belief to hold. Sometimes people profess ridiculous things, but very few of them put their money where their mouth is on prediction markets. [1]
1. I’ve seen some pretty mispriced markets. At one point in 2019, PredictIt had Andrew Yang at 16% to win the Democratic presidential primary. And in 2020, Donald Trump was about 16% to become president even after he had lost the election. But the sorts of people who bet on prediction markets are not the sorts of fundamentalist Christians who think that Jesus Christ has a high chance of returning this year.
yes, no one would put a large amount of money (say, $10,000) on let’s say a 1-year time horizon, “joe biden going to prison”, “barack obama going to prison”, “nancy pelosi, bill clinton, and hillary clinton going to prison”, “trump being put in office prior to the 2024 election”, and if someone did make such a bet, they wouldn’t be motivated by listening to a christian minister who regularly makes political / religious prophecies. surely no one would do that.

i don’t know why anyone who posts on a forum devoted to outright fringe beliefs and atypical personality traits (i say with all love and kindness, not to say that any of us are bad or incorrect for these beliefs, merely that they are objectively abnormal) is going to come out and make bold claims that there exists no such weirdo who is willing to do X for Y reasons.

the main point about the time value of money is interesting enough on it’s own, but the interesting, nerd-crack explanation is probably just not true. there are probably just crazy people who bet on the return of jesus christ.

casens 13 Mar 2025 3:44 UTC
1 point
−1
in reply to: nostalgebraist’s comment on: when will LLMs become human-level bloggers?
this is a fair response, and to be honest i was skimming your post a bit. i do think my point somewhat holds, that there is no “intelligence skill tree” where you must unlock the level 1 skills before you progress to level 2.

i think a more fair response to your post is:
1. companies are trying to make software engineer agents, not bloggers, so the optimization is towards the former.
2. making a blog that’s actually worth reading is hard. no one reads 99% of blogs.
3. i wouldn’t act so confident that we aren’t surrounded by LLM comments and posts. are you really sure that everything you’re reading is from a human? all the random comments and posts you see on social media, do you check every single one of them to gauge if they’re human?
4. lots of dumb bots can just copy posts and content written by other people and still make an impact. scammers and propagandists can just pay an indian or philipino $2/hr and get pretty good. writing original text is not a bottleneck.

casens 12 Mar 2025 15:22 UTC
0 points
1
on: when will LLMs become human-level bloggers?
Surely it would be exceptionally good at those kinds of writing, too, right?
surely an LLM capable of writing A+ freshman college papers would correctly add two 2-digit numbers? surely an AI capable of beating grandmasters in chess would be able to tutor a 1000 elo player to a 1500 elo or beyond? surely an AI capable of answering questions at a university level in diverse subjects such as math, coding, science, law, would be able to recursively improve itself and cause an intelligence explosion? surely such an AI would at least be able to do a simple task like unload a dishwasher without breaking a dish?

i think surely it should be obvious to anyone who’s able to mull it through for a few seconds that intelligence does not need to progress along the same paths as it does for human civilization over centuries or for human beings through child development, or even among proto-intelligent animals on earth. it is surely obvious to me that AI can exhibit surprising mixes of general and non-general intelligence, and that we’re not really sure why it works. there is really no requirement i have left, that before the AI turns us into paperclips, it must 100% be able to beat poker players at the WSOP or generate an oscar-winning feature film, or be able to make nobel-winning science discoveries. some of these requirements seem more plausible than others, but none seem totally certain.

casens 16 Nov 2024 15:46 UTC
LW: 5 AF: 4
2
AF
on: o1 is a bad idea

In some sense, the Agent Foundations program at MIRI sees the problem as: human values are currently an informal object. We can only get meaningful guarantees for formal systems. So, we need to work on formalizing concepts like human values. Only then will we be able to get formal safety guarantees.

unless i’m misunderstanding you or MIRI, that’s not their primary concern at all:

Another way of putting this view is that nearly all of the effort should be going into solving the technical problem, “How would you get an AI system to do some very modest concrete action requiring extremely high levels of intelligence, such as building two strawberries that are completely identical at the cellular level, without causing anything weird or disruptive to happen?”

Where obviously it’s important that the system not do anything severely unethical in the process of building its strawberries; but if your strawberry-building system requires its developers to have a full understanding of meta-ethics or value aggregation in order to be safe and effective, then you’ve made some kind of catastrophic design mistake and should start over with a different approach.

casens 29 Apr 2023 5:06 UTC
6 points
3
in reply to: paulfchristiano’s comment on: Transcript and Brief Response to Twitter Conversation between Yann LeCunn and Eliezer Yudkowsky
this was posted after your comment, but i think this is close enough:
@ylecun
And the idea that intelligent systems will inevitably want to take over, dominate humans, or just destroy humanity through negligence is preposterous.
They would have to be specifically designed to do so.
Whereas we will obviously design them to not do so.