Karl Krueger

Karma: 816

Karl Krueger 16 Dec 2025 2:49 UTC
1 point
−1
on: Why do LLMs so often say “It’s not an X, it’s a Y”?
My guess is that they do so in imitation of humans who do the same thing when asked the sorts of questions that people ask LLMs. It’s not an LLM thing; it’s a thing one does to make distinctions clear, when the other person might otherwise conflate two distinct entities, clusters, or topics. It just so happens that people ask LLMs a lot of that sort of question, and thus elicit a lot of that particular response.
(I also use em dashes, yes.)

Karl Krueger 16 Dec 2025 2:18 UTC
3 points
0
on: Viewing animals as economic agents
The equivalent in animals is dopamine: there is no amount of dopamine beyond which an animal would prefer not to get more dopamine.
Dopamine is the brain’s universal signal that something an animal did was good for it.
My understanding is that dopamine signaling is more involved in anticipation of an outcome, to engage the process that coordinates action towards or away from that outcome, rather than in experiencing a reward of having reached a positive outcome. Notably, dopamine is involved in anticipation of negative outcomes as well as positive ones!
Wikipedia (emphasis added)—
In popular culture and media, dopamine is often portrayed as the main chemical of pleasure, but the current opinion in pharmacology is that dopamine instead confers motivational salience; in other words, dopamine signals the perceived motivational prominence (i.e., the desirability or aversiveness) of an outcome, which in turn propels the organism’s behavior toward or away from achieving that outcome.

Karl Krueger 15 Dec 2025 17:47 UTC
5 points
0
on: How to account for misinformation when looking for effective altruist causes?
Transitive trust: If you trust Alice, and Alice trusts Bob (of Bob’s Discount Bed Nets & Vaccine Research Shop), then you might trust Bob somewhat on the strength of Alice’s trust.
Transitive authentication (weaker): If you believe Alice is a real person and not a spambot (because you’ve met her), and Alice believes Bob is a real person and not a spambot, then you might trust that Bob is not a spambot too.
Track record: If the Carter Center has been effective against guinea worm, then you have reason to believe they’ll be effective against other horrible parasitic diseases in the future. (This establishes that they’re competent, not that they’re the optimal cause on margin.)
Reasoned community discourse: If you observe open discussion of causes, where people actively disagree with one another in good ways, change one another’s mind in ways that are consistent with intelligent evaluation, including calling out mistakes or bad reasoning, you might credit the evidence presented in that discourse. (The disagreement and calling-out of mistakes are necessary to make sure you’re not looking at a mutual admiration society.)
Bay Area House Party: Go hang out with a bunch of EAs and troll them into an argument over whose cause is better. (If they all agree, find a different party.)

Karl Krueger 13 Dec 2025 20:40 UTC
12 points
0
on: You Can Just Buy Far-UVC
https://www.nature.com/articles/s41598-025-09241-2
Corneal safety assessment of germicidal far UV-C radiation
Abstract
Far UV-C radiation (200–240 nm) is a promising alternative to conventional UV-C for disinfection in occupied spaces, offering strong germicidal efficacy with reduced skin risk. However, its ocular safety remains unclear, as most studies relied only on non-human corneal models with physiological differences. This study investigated UV-induced DNA damage in the epithelium, stroma, and endothelium of ex vivo human corneas and porcine corneas, and reconstructed human cornea epithelium (RHCE) using immunohistochemistry. Samples were exposed to 222 nm, 233 nm, 254 nm, and broadband UV-B (280–400 nm) radiation in the presence of real human tears. Compared to human corneas (26 μm mean epithelium thickness), porcine corneas (110 μm) and RHCE (79 μm), showed reduced UV penetration. In human corneas with a thin epithelium, far UV-C exposure led to epithelial and anterior stromal damage, underscoring the epithelium’s protective function. Optical properties using porcine corneas confirmed the immunohistological findings, validating wavelength-dependent penetration depths. Simulations suggest that in intact human corneas, damage-relevant intensity of 222 nm light reaches the middle of the epithelium, while for 233 nm, it reaches the basal layer. These findings support the relative safety of far UV-C, especially 222 nm, for intact corneas. However, potential DNA damage accumulation after repeated exposures underscores the need for further research on long-term ocular effects.

Karl Krueger 10 Dec 2025 1:22 UTC
6 points
0
in reply to: Mateusz Bagiński’s comment on: Mateusz Bagiński’s Shortform
Russell and Norvig discuss “intelligent agents” in AIMA (2003) and they don’t mean web scrapers or database scripts, but they also don’t mean that the thing they’re discussing is conscious or super-rational or anything fancy like that. A self-driving car is an “agent” in their sense.
I suspect the use of “agentic” to mean something like “highly instrumentally rational” — as in “I want to become more agentic” — is an LW idiosyncrasy.
In human psychology, Milgram used “agentic” to mean “obedient”, in contrast to “autonomous”!
As an aside, the origins of “LGBT” and “racism” are not quite what you say. A historical dictionary may help. “LGBT” was itself an expansion of earlier terms. LGB (and GLB) were used in the 1990s; and LG is found in the 1970s, for instance in the name of the ILGA which was originally the International Lesbian & Gay Association and more recently the International Lesbian, Gay, Bisexual, Trans and Intersex Association while retaining the shorter initialism.

Karl Krueger 9 Dec 2025 4:30 UTC
2 points
1
in reply to: StartAtTheEnd’s comment on: The Possibility of an Ongoing Moral Catastrophe
No, evil does not become good just because you’re bored.

Karl Krueger 8 Dec 2025 23:09 UTC
1 point
0
in reply to: robertzk’s comment on: robertzk’s Shortform
One specific practice that would prevent this:
Tutorials or other documentation that need example IPv4 addresses should choose them from the RFC 5737 blocks reserved for this purpose. These blocks are planned to never be assigned for actual usage (including internal usage) and are intended to be filtered everywhere at firewalls & routers.
- 192.0.2.0 − 192.0.2.255
- 198.51.100.0 − 198.51.100.255
- 203.0.113.0 − 203.0.113.255

Karl Krueger 7 Dec 2025 17:31 UTC
11 points
0
on: Lawyers are uniquely well-placed to resist AI job automation
We know that some lawyers are very willing to use LLMs to accelerate their work, because there have been lawyers caught submitting briefs containing confabulated case citations. Probably many other lawyers are using LLMs but are more diligent about checking their output — and thus their LLM use goes undetected.
I wonder if lawyering will have the same pipeline problem as software-engineering: The “grunt work” that has previously been assigned to trainees and junior professionals will be automated early on; thus making it less valuable to hire juniors; thus making it harder for juniors to gain job experience.
(Though the juniors can be given the task of manually checking all the citations …)

Karl Krueger 5 Dec 2025 23:03 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: Richard_Kennaway’s Shortform
Sure, but the boss can go wrong by creating an incentive structure in which questioning a message “from the boss” is unsafe.
Successful anti-phishing campaigns instill not only doubt (“Is this actually from the boss?”) but also permission to act on that doubt (“I’ve got the boss’s cell phone number already; when I’m not sure if the message is from the boss, I’m supposed to call the boss and check, with no chance of bad consequences for pestering her.”)

Karl Krueger 5 Dec 2025 20:33 UTC
11 points
0
in reply to: Axel Ahlqvist’s comment on: Eval-unawareness ≠ Eval-invariance
It seems worth pointing out that humans may be eval-unaware but we don’t want to be. The simulation hypothesis and the existence of an afterlife are things that humans do care about. If we could find out for sure whether we’re in a simulation, most people would want to. We do care about whether the world we live in is just a morality-test to gain access to the Good Place.
Humans aren’t eval-invariant either. Humans who believe in a morality-segregated afterlife don’t tend to be more moral than humans who believe death is the end; but they do differ in some observable behaviors. (A cynical atheist might say they are more sycophantic.)

Karl Krueger 5 Dec 2025 19:40 UTC
1 point
0
in reply to: Richard_Kennaway’s comment on: Richard_Kennaway’s Shortform
Another contributing factor might be a person’s level of anxiety about the DVLA, or government, or email, or “the system”. When people are anxious about what “the system” might do to them, and prepared for it to make novel demands upon them, that primes them to be scammable.
A different example: If you want to phish an office-worker, one way to do it is to pretend to be their boss and make sudden urgent demands of them. If the office-worker fears that they will be fired if they don’t comply with novel demands from their boss, then they are primed to be scammable. Workers who feel unsafe questioning “their boss’s” orders will be more scammable than workers who feel safe calling bullshit on their actual boss once in a while.

Karl Krueger 5 Dec 2025 19:27 UTC
5 points
0
on: Who is AGI for, and who benefits from AGI?
Currently, AGI is mostly being developed by human engineers and scientists within human social systems. [...] There are far fewer literature professors, historians, anthropologists, creatives, social workers, landscape design architects, restaurant workers, farmers, etc., who are intimately involved in creating AGI. This isn’t surprising or illogical, but if AI is likely to be useful to “everyone” in some way (à la radio, computers), then “everyone” probably needs to be involved.
This concern seems somewhat misdirected.
There weren’t a lot of landscape design architects or farmers involved in the development of radio or computers. It was done by engineers, product managers, technology hobbyists, research scientists, logicians, etc.; along with economic demand from commerce, military, and other users capable of specifying their needs and paying the engineers etc. to do it.
Were landscape architects excluded from developing radio? Did anyone prevent farmers from developing computers? No, they were just busy doing landscape design and farming. Eventually someone built computer systems for architects and farmers to use to get more architecting and farming done.
And then the product managers and sales people made sure that they charged the architects and farmers a butt-ton of money. Downstream of that is why both the farmers and the open-source folks have a problem with John Deere’s licensing and enforcement practices; and the architects ain’t particularly thrilled by Autodesk’s behavior either.
You can’t align AGI with the CEV of engineers to the exclusion of other humans, because engineers are not that different from other humans. That’s not the problem. But aligning AGI with “number go up” to the exclusion of other human values, that’s a problem. Even people who like capitalism don’t tend to believe that capitalism is aligned with all human values. That’s something to worry about.

Karl Krueger 1 Dec 2025 19:01 UTC
3 points
0
on: 10 Aphorisms from 𝘛𝘩𝘦 𝘉𝘦𝘥 𝘰𝘧 𝘗𝘳𝘰𝘤𝘳𝘶𝘴𝘵𝘦𝘴
Small injustice dresses up as vice; large injustice dresses up as virtue.
This is dated. Vice signaling has become a central element of the public image of many perpetrators of large injustice.
The problem of knowledge is that there are many more books on birds written by ornithologists than books on birds written by birds and books on ornithologists written by birds.
Almost zero species do any writing (or, indeed, knowing) at all.

Karl Krueger 30 Nov 2025 20:17 UTC
3 points
3
on: Should you work with evil people?
Sometimes, working with evildoers entails providing them with access to victims that they otherwise wouldn’t get their hands on.
Sometimes it comes to involve defending the evildoers against criticism; casting doubt on true warnings that others give about the evildoers’ conduct; or providing reputational cover for them.
If the evildoers are really clever, they may find a way to describe criticism or warnings as “abuse of the infrastructure” and try to get their critics banned under the principle expressed in your conclusion.

Karl Krueger 29 Nov 2025 19:24 UTC
1 point
0
on: I gave LLMs emotional damage
Does this work any differently for emotions than, for say, bridges or dogs or any other particular meaning?

Karl Krueger 28 Nov 2025 3:46 UTC
3 points
0
in reply to: Brendan Long’s comment on: Robust Software Isn’t About Error Handling
Or you may be doing a rollout of a new abcserver, and the monitoring system tracks that the abcserver itself is successfully starting and taking queries … but not noticing that the new abcserver’s responses are causing the jklserver (somewhere else in your infra) to crash.
One possible rule is, “If it looks like there’s a global sitewide outage happening, look up all the currently in-progress and recently-completed rollouts. Send an alert to all of those teams. Everyone who’s responsible for a recent rollout gets to ponder deep in their hearts whether there’s any possibility that their rollout could be breaking the world, and decide whether to roll back.”
Another possible rule is, “You can afford to have an agent (currently, probably a human) watching over each rollout; an agent who cares about whether the whole site is crashing hard, and who gets curious about whether their rollout might be responsible.”

Karl Krueger 28 Nov 2025 1:14 UTC
3 points
2
on: Robust Software Isn’t About Error Handling
As I read it, the interesting part isn’t around the unwrap operation. The interesting part is that a new version was being pushed out incrementally, and the infrastructure components that read from the new version were failing consistently. But the awareness of the failure did not propagate back into the component doing the push.
The point of an incremental rollout is to allow a fraction of traffic to be affected by the new version, as a test of whether the new version is actually good. In order for that to work, failures caused by the new version have to be detected; and that awareness has to propagate back to the system doing the rollout, and cause the rollout to stop.

Karl Krueger 28 Nov 2025 1:08 UTC
3 points
0
on: The First Thanksgiving
A bit of religious history: The Pilgrims of Plymouth Colony are often described as Puritans. They were not Puritans.
The Massachusetts Bay Colony (centered in Salem and later in Boston) were Puritans. Plymouth Colony (in Plymouth and Cape Cod) were Separatists. The two colonies did not merge until 1691, fully three score and eleven years after the founding of Plymouth Colony.
The original ideological difference between Puritans and Separatists was that, back in England, the Puritans had originally aimed at “purifying” or reforming the established Church of England, that is, taking over an existing power structure; whereas Separatists had aimed at establishing a separate church, declaring independence from that power structure.

Karl Krueger 27 Nov 2025 21:06 UTC
2 points
0
in reply to: bfinn’s comment on: bfinn’s Shortform
Hmm …
Most people use the same shower every day, so they only have to learn a new shower UI when they move to a different house (or are visiting someone, etc.) Ease of first use and ease of habitual use are not the same target. Once a user has learned a particular shower UI, they can probably operate it half-asleep.
Once installed, a shower stays in operation a long time. I’ve used showers that were 50+ years old in the past year. (Contrast with cars, which have an average lifespan of 12 years.) So even if there was some massively better shower UI developed today, that was so great that 100% of new showers used it, for most people it would remain unfamiliar for a long time.
Because of the long lifespan of showers, any “beneficial mutation” (UI improvement) would take a long time to reach fixation. So showers evolve slowly, aesthetic design trends may overrule functional improvements, and knowledge may get lost: maybe the best shower UI design of 50 years ago is not available today because it was tied to a particular 1970s aesthetic.

Karl Krueger 26 Nov 2025 18:41 UTC
1 point
0
in reply to: green_leaf’s comment on: Should we extend moral patienthood to LLMs?
It makes sense that animals evolved the capacity to experience pain and suffering, because we have bodies that can be injured, starved, sickened, and so on. There are stimuli that correctly identify threats to our well-being; and so we have developed to perceive those stimuli as noxious and well-worth-avoiding. But this suggests that a mind that developed without such threats would not need the capacity to suffer, just as a fish that lives in a pitch-black cave does not need the capacity to see.

Karl Krueger

Corneal safety assessment of germicidal far UV-C radiation

Abstract