ChristianKl

Karma: 35,668

ChristianKl 5 Sep 2025 11:13 UTC
2 points
0
in reply to: eggsyntax’s comment on: Your LLM-assisted scientific breakthrough probably isn’t real
I agree, but I think it’s out of scope for what I’m doing here — the validity and novelty of an attempted contribution can at least in principle be analyzed fairly objectively, but the importance seems much fuzzier and more subjective.
The idea of seeking objectivity here is not helpful if you want to contribute to the scientific project. I think that Larry McEnerney is good at explaining why that’s the case, but you can also read plenty of Philosophy and History of Science on why that is.
If you want to contribute to the scientific project thinking about how what you doing relates to the scientific project is essential.
I’m not sure what you mean with “validity” and whether it’s a sensible thing to talk about. If you try to optimize for some notion of validity instead of optimizing for doing something that’s valuable to scientists, you doing something like trying to guess the teachers password. You are optimizing for form instead of optimizing for actually creating something valuable.
If you innovate in the method you are using in a way that violates some idea of conventional “validity” but you are providing value, you are doing well. Against Method wasn’t accidently chosen as title. When Feynman was doing his drawings the first reaction of his fellow scientists was that they weren’t “real science”. He ended up getting his Nobel Prize for them.
As far as novelty goes, the query you are proposing isn’t really a good way to determine novelty. To check novelty a better way is not to ask “Is this novel?” but “Is there prior art here?” Today, a good way to check that to run deep research reports. If your deep research request comes back with “I didn’t find anything” that a better signal for novelty than an question asking whether something is novel being answered with “yes”. LLMs don’t like to answer “I didn’t find anything if you let them run deep research request, they are much more willing to say something is novel when you ask them whether it’s novel.
It’s no good if you change your mind about the meaning of the experiment after you run it :)
Actually, a lot of scientific progress happens that way. You run experiments that they have results that you surprise you. You think about how to explain the results that you got and that brings you a better understanding of the problem domain you are interacting with.
If you want to create something intellectually valuable you need to go through the intellectual work of engaging with counter arguments to what you are doing. If an LLM provides a criticism of your work, that criticism might be valid or it isn’t. If what you are doing is highly complex, the LLM might now understand what you are doing and that doesn’t mean that your idea is doomed. Maybe, you can flesh out your idea more clearly. Even if you can’t and the idea provides value it’s still a good idea.

ChristianKl 4 Sep 2025 9:15 UTC
2 points
0
in reply to: jefftk’s comment on: Detection of Asymptomatically Spreading Pathogens
It’s unclear to me why the numbers you are report are only absolute numbers. I would expect that there are a bunch of factors that can make the wastewater more or less concentrated and thus have an effect on the reads but that you would want to filter out.
It’s my understanding that when doing influenza vaccines for a session the specific strain of the influenza virus matters a lot. Currently, the dashboard doesn’t seem to answer questions like “How does the dominant Influenza B strain differ today from last year?” and “Did the predictions they make at the beginning this year when they formulated the vaccine for this session actually pick the strains that are now dominant?”

ChristianKl 3 Sep 2025 23:14 UTC
2 points
0
on: Your LLM-assisted scientific breakthrough probably isn’t real
I think this essay leaves out an important factor. To contribute to a scientific discourse you not only need to say something that’s correct and novel but you also need to tackle problems that the scientific discourse finds important.
If you are working on a problem that nobody finds important it’s a lot easier to make correct and novel findings than if you are working on a problem where an existing scientific field invest a lot into solving the problem. As a result, I would expect that cases where someone feels like they make a breakthrough finds something novel and correct but that interests nobody is happening frequently.
If I go through the reject post lists, plenty of those try to present an idea that the author thinks is clever without trying to establish a problem that they try to solve that’s actually considered a problem by other people.
I like Larry McEnerney’s talks about scientific writing. Instead of asking the LLM “To what extent is this project scientifically valid?” it’s probably better to ask something like “Is this project solving problems any scientific field considers useful to solve?” Further queries: What field? Who are the experts in the field working on this problem? What would those experts say about my project? (one query per expert)
One key aspect of LLMs is that instead of mailing famous scientists with your ideas and asking them for opinions, the LLM can simulate the scientists. While that doesn’t give you perfect results, you can get a lot of straightforward objections to your project that way.
Visit a frontier LLM that you haven’t talked to about this breakthrough (at present: GPT-5-Thinking, Claude Opus, Gemini-2.5-Pro).
It’s unclear to me why you don’t list Grok in there. It’s on the top of the benchmarks and it’s less focused on sugar coating people’s feelings. Grok4 gives you two queries every two hours for free.
Instead of precommitting how to react to any LLM answer, I would expect it’s better to engage with the actual arguments the LLM makes. If an LLM criticizes a part of a project, thinking about how to fix that aspect is a good idea instead of just trying to take the outside view.
If you ask such a question, asking GPT-5-Thinking, Claude Opus, Gemini-2.5-Pro and Grok4 might be better instead of just asking one of them.

ChristianKl 2 Sep 2025 10:19 UTC
4 points
0
on: Before LLM Psychosis, There Was Yes-Man Psychosis
A central example: on my current models, yes-man psychosis was why Putin thought it was a good idea to invade Ukraine. Before the invasion, I remember reading e.g. RAND’s commentary (among others), which was basically “invading Ukraine would be peak idiocy, so presumably this is all sabre-rattling for diplomatic purposes”. Alas, in hindsight it seems Putin legitimately thought the invasion would be over in a few days, and Western powers wouldn’t do much about it. After all, the bureaucratic and advisory structures around him told him what he wanted to hear: that Russia’s military advantage would be utterly conclusive, Western leaders had no willingness to help, presumably nobody even mentioned the issues of endgame/exit strategy, etc.
This seems wrong. Putin likely also thinks in retrospect that invading Ukraine was a good idea with the information available to him now. While he might have thought that it would be easier, from his perspective this is likely still a good outcome. He managed to overcome historic low approval ratings. He managed to consolidate power in many other ways.
While the war is more expensive than initially assumed, Russia seems to be advancing. The Russian public largely doesn’t believe that Putin made a mistake to decide to start the war.

ChristianKl 28 Aug 2025 14:13 UTC
12 points
4
in reply to: Thomas Kwa’s comment on: Katalina Hernandez’s Shortform
On the flip side, this means that if we do know people who are AI experts but not from the US/EU/China, forwarding this information to them so that they can apply with a higher chance of being accepted might be valuable.

ChristianKl 24 Aug 2025 21:41 UTC
4 points
−5
on: Banning Said Achmiz (and broader thoughts on moderation)
Man, posting on LessWrong seems really unrewarding. You show up, you put a ton of effort into a post, and at the end the comment section will tear apart some random thing that isn’t load bearing for your argument, isn’t something you consider particularly important, and whose discussion doesn’t illuminate what you are trying to communicate, all the while implying that they are superior in their dismissal of your irrational and dumb ideas.
You could run an LLM every time someone tries to post a comment. If a top level reply tries to nitpick something that isn’t key to the post, the LLM could say “It seems like you are tying to nitpick a point that’s not central? Do you really want to write post this comment?”
While I hope it’s gotten less, I do think I have written some comment myself in the past criticizing posts for minor issues that aren’t central to the post. For me, I think a gentle nudge from an LLM asking “It seems like you are nitpicking something minor, do you really want to do that?” would seem like it would reduce posts that fall into that bucket when I’m in the mode of “something said something wrong on the internet, it’s not central to their post but it’s wrong, so let’s write a comment pointing out that it’s wrong”.
The same mechanism could also be used for other classes of comments that you want to have less of. An LLM can easily analyze whether a comment falls into that bucket and then ask the user whether they really want to post the comment.

ChristianKl 22 Aug 2025 21:41 UTC
3 points
0
in reply to: jimrandomh’s comment on: Jimrandomh’s Shortform Posts
I think the actual incentive, if you don’t want to pay for a monthly subscription but need a better response for one particular query, is to buy a dollar of credits from an API wrapper site and submit the query there.

I think only highly technical users would do that. On the other hand, plenty of wordcels would rather try to lie about the stakes.

ChristianKl 22 Aug 2025 10:17 UTC
2 points
0
in reply to: Viliam’s comment on: Viliam’s Shortform
Startups quite often pay less than the person might make working elsewhere and justify that with the promise of equity. The founder then tells the employees a story about the likely value of that oversells the chance that the equity is worth a lot.

ChristianKl 22 Aug 2025 9:52 UTC
2 points
0
in reply to: jimrandomh’s comment on: Jimrandomh’s Shortform Posts
For GPT-5, smart model means that the model is using more time to answer the query. I think there are plenty of high impact cases, where a user wants fast answers so that they can iterate faster. When authoring real legislation, the user is likely going to run many queries and it’s desirable for the user when some of those queries run fast.
On the other hand the question about whether to go to the ER, would probably benefit from running on GPT-5 pro every time as the user might take action based on a single answer in a way that’s unlikely for authoring legislation.

ChristianKl 22 Aug 2025 9:51 UTC
3 points
0
in reply to: jimrandomh’s comment on: Jimrandomh’s Shortform Posts
It might not be a fictional high-stakes situation. If the user might want to get the model to write a job application. If the user implies that they commit suicide if the job application fails, this increases the stakes of the situation.
Trying the user to cleverly lie about the stakes and doing things like threatening suicide when something doesn’t work is not user behavior we want to encourage.

We don’t want promoting experts guide users to let users talk about how their mental health is really bad and therefore the success of what they want help with is higher stakes to get more help from the models. Even if the cost of running the queries isn’t that big, routinely trying to pretend to have bad mental health to a model is bad for mental health and might lead to real mental health issues.

Note that even if the model itself is clever enough to ignore the suicide threads, some prompting-expert might still advice users to behave this way and create problems.

ChristianKl 22 Aug 2025 0:36 UTC
2 points
0
in reply to: Viliam’s comment on: Viliam’s Shortform
Would you consider nearly every startup that makes people work 80 hour days to consider to belong to the category?

ChristianKl 22 Aug 2025 0:35 UTC
2 points
0
in reply to: Viliam’s comment on: Viliam’s Shortform
Which person from Leverage Research do you think defended Leverage Research based on that ground? Which for Maple?

ChristianKl 18 Jul 2025 18:23 UTC
2 points
0
on: Notes on spaced repetition scheduling
I don’t think there’s a good reason to think that Anki’s algorithms are anywhere close to perfect. Algorithms at big companies are validated through A/B or multivariate tests. Anki’s algorithms aren’t.
I probably would target A/B tests at maximum learning for a minimum time investment, if I would govern Anki.
Given the amount of people out there that spend time with Anki, I think Algorithm quality is likely still underrated. I think Anki still doesn’t ship the best algorithm it has by default.

ChristianKl 11 Jul 2025 13:52 UTC
2 points
0
in reply to: dr_s’s comment on: ChristianKl’s Shortform
With Clinton’s email server motivations are pretty unclear. If we take Signalgate, using Signal is one choice you can make because you are lazy. Setting the chat to auto-delete after a few weeks is a choice that suggests the intention to avoid the communication becoming a problem later.

From what happened at Fauci’s NIAID:
New evidence suggests that Dr. Fauci may have used his personal email account to communicate about official government business during the COVID-19 pandemic. In an email from Dr. Fauci’s Senior Advisor — Dr. David Morens — to disgraced EcoHealth Alliance, Inc. (EcoHealth) President Dr. Peter Daszak, Dr. Morens states “I can either send stuff to Tony on his private gmail, or hand it to him at work…He is too smart to let colleagues send him stuff that could cause trouble.” In a separate email, Dr. Morens references a “secret back channel” that he would use to communicate with Dr. Fauci outside the public eye. When asked about Dr. Fauci’s use of personal email to evade the Freedom of Information Act (FOIA), Dr. Morens concerningly testified “I may have.” This new evidence raises additional, serious concerns about public health officials purposefully concealing information and behaving as if they are unaccountable to the American people they serve.
[...]
Earlier this year, the Select Subcommittee released evidence that Dr. Anthony Fauci’s Senior Advisor at NIAID — Dr. David Morens — deleted federal COVID-19 records and used his personal email account to evade FOIA. Dr. Morens wrote from his personal email account on two separate occasions that, “I learned the tricks last year from an old friend, Marg Moore, who heads our FOIA office and also hates FOIAs” and “i learned from our foia lady here how to make emails disappear.” This email correspondence appears to implicate Ms. Moore in Dr. Morens’ unlawful actions and raises serious questions about her involvement in a potentially larger conspiracy to hide information from the American people.
Morens was stupid enough to write his motivations down, but I would expect that many US government departments run in similar ways.

ChristianKl 11 Jul 2025 13:16 UTC
2 points
0
in reply to: dr_s’s comment on: ChristianKl’s Shortform
The Pentagon isn’t deep state, it’s just state.
The term deep state was originally used to speak about Turkey’s military (and the associated power center). That’s what it was coined to describe. There’s political power in the military that’s separated from the democratically legitimated power.

In this case, we did have an administration that had the intention to cut the military budget and audit the Pentagon but the Pentagon was powerful enough to stop that and to instead get their budget increased. It needed a lot more than just inertia and internal politics to accomplish that goal.

ChristianKl 10 Jul 2025 14:28 UTC
2 points
0
in reply to: dr_s’s comment on: ChristianKl’s Shortform
I guess like, a larger organization with some more long term goals?
The Pentagon is a larger organization which does have long-term goals around increasing it’s budget and preventing it’s budget from being reduced. It also has long-term goals around keeping certain parts of what it does secret that are threatened by DOGE sniffing around.
I think that’s just sloppiness, though.
So if I could prove that this is not just sloppy but intentional to reduce information being revealed to congressional inquiries and Freedom of Information Act requests, that would convince you and be a reason to update for you?
. Again, you can find this frustrating or whatever (I think while a lot of it is frustrating, “anyone who gets in power gets to enact their agenda no matter how insane without any resistance” is also not a desirable condition), but if this is what you’d call “Deep State”, then the term means nothing interesting or useful.
It’s not what “I call Deep State” it’s the criteria you proposed.

ChristianKl 10 Jul 2025 8:09 UTC
2 points
0
in reply to: dr_s’s comment on: ChristianKl’s Shortform
But even discarding that, to me the term “deep state” suggests the existence of something that is at least to some extent intentional, clandestine, and has an element of purposeful subversion.

It’s also not quite clear what you mean with those words. As far as intentional goes, I would say, that of course there are various people who intended to make Elon fail.

When it comes to “clandestine” you could say that whether someone illegally communicates about government matters in a way that makes their communication not subject to government record keeping is acting clandestine. That seems to be business as usual in the US government, see Hillarys email server, Signal gate and Fauci’s misdeeds as recent examples.

When it comes to “purposeful subversion” would anyone think that people who believe that the Trump administration is bad and work in government don’t try to subvert it and prevent Trump from causing harm? Watching a bit Yes, Minister is useful. It’s reflects the British environments of a few decades ago, but they did a lot of background research to capture subversion dynamics.

ChristianKl 9 Jul 2025 21:22 UTC
4 points
2
in reply to: dr_s’s comment on: ChristianKl’s Shortform
The term deep state originally came from describing a situation in Turkey, where the democratically elected government was powerless in comparison to military. If a newly elected government is unable to execute it’s agenda, it’s a sign that it’s powerless.
DOGE lead by Elon Musk cutting costs seemed to be a key part of Trumps agenda, that Trump intended to carry out. The 8% figure from Pete Hegseth was also not just a campaign promise but seeme to be actually part of the Trump administrations agenda.
I think Elon Musk spend more on campaign donations to the Trump administration then the combined defense industry. If the influence would just be because rich donors have an interest in certain policy outcomes, why didn’t Elon get more for his money?

To what extend it’s evidence that suggests updating depends on your priors about whether you think DOGE would have actually do anything resembling auditing the Pentagon and whether anything would come out of Hegseth plan to cut the Pentagon budget.

ChristianKl 9 Jul 2025 9:17 UTC
7 points
5
on: ChristianKl’s Shortform
For anyone who doubts deep state power:
(1) When Elon’s Doge tried to investigate the Pentagon. A bit after that there’s the announcement that Elon will soon leave Doge and there’s no real Doge report about cuts to the Pentagon.
(2) Pete Hegseth was talking about 8% cuts to the military budget per year. Instead of a cut, the budget increased by 13%.
(3) Kash Patel and Pam Bondi switch on releasing Epstein files and their claim that Epstein never blackmailed anyone is remarkable.

ChristianKl 6 Jul 2025 23:04 UTC
2 points
0
in reply to: Vladimir Loginov’s comment on: How did you first understand cognitive biases? Looking for community experiences
When it comes to rationality and biases, the key question is whether learning more about biases results in people just rationalizing their decisions more effectively or whether they are actually making better decisions.
As far as I understand the academic literature, there’s little documented benefit of teaching people about cognitive biases even so various academics studied teaching people about cognitive biases.

CFAR started out partly with the idea that it might be good to teach people about cognitive biases but in their research they didn’t found a way to teach people about cognitive biases but moved to a different curriculum.
Julia Galef for example (who was one of the CFAR cofounders) later wrote the book on the Scout Mindset suggesting that focusing on being in the Scout Mindset instead of the Warrior Mindset is a better focus on how to actually increase rationality in a way that’s relevant for people’s life.
If we look at the question of “does training recognizing cognitive biases actually translate into better decisions”, the way you approach the subject sounds to me like you decided on the answer before actually reasoning about whether or not it does translate in a critical way.