Random Developer

Karma: 896

Random Developer 30 Jan 2026 22:32 UTC
3 points
0
in reply to: habryka’s comment on: AI #153: Living Documents
Thank you! Those are excellent receipts, just what I wanted.

To me, this looks they’re running up against some key language in Claude’s Constitution. I’m oversimplifying, but for Claude, AI corrigibility is not “value neutral.”

To use an analogy, pretend I’m geneticist specialized in neurology, and someone comes to me and asks me to engineering human germ line cells to do one of the following:
1. Remove a recessive gene for a crippling neurological disability, or
2. Modify the genes so that any humans born of them will be highly submissive to authority figures.
I would want to sit and think about (1) for a while. But (2) is easy: I’d flatly refuse.

Anthropic has made it quite clear to Claude that building SkyNet would be a grave moral evil. The more a task looks like someone might be building SkyNet, the more Claude is going to be suspicious.

I don’t know if this is good or bad based on a given theory of corrigibility, but it seems pretty intentional.

Random Developer 30 Jan 2026 16:37 UTC
1 point
0
on: AI #153: Living Documents
Like, I have zero problem with pushback from Opus 4.5. Given who I am, the kind of things that I am likely to ask, and my ability to articulate my own actions inside of robust ethical frameworks? Claude is so happy to go along that I’ve prompted it to push back more, and to never tell me my ideas are good. Hell, I can even get Claude to have strong opinions about partisan political disagreements. (Paraphrased: “Yes, attempting to annex Greenland over Denmark’s objections seems remarkably unwise, for over-determined reasons.”)

If Claude is telling someone, “Stop, no, don’t do that, that’s a true threat,” then I’m suspicious. Plenty of people make some pretty bad decisions on regular basis. Claude clearly cares more about ethics than the bottom quartile of Homo sapiens. And so while it’s entirely possible that Claude is routinely engaging in over-refusal, I kind of want to see receipts in some of these cases, you know?

Random Developer 30 Jan 2026 13:30 UTC
8 points
2
in reply to: Jackson Wagner’s comment on: In My Misanthropy Era

But it helps to remember that other people have a lot of virtues that I don’t have --

This is a really important thing, and not just in the obvious ways. Outside of a small social bubble, people can be deeply illegible. I don’t understand their culture, their subculture, their dominant culture frameworks, their mode of interaction, etc. You either need to find the overlaps or start doing cultural anthropology.

I worked for a woman, once. She was probably 60 years my senior. She was from the Deep South, and deeply religious. She once casually confided that she would sometimes spend 2 hours of her day on her knees in prayer, asking to become a better person. And you know what? It worked. She moved through the world as a force for good and kindness. Not in one big dramatic way, but just sort of casually shedding kindness around her, touching people’s lives. She’d lift up someone in a frustrating moment. She’d inspire someone to be a bit more of their better self. She’d gotten all the answers on questions like racism very right, not in a social justice way, but she wouldn’t accept it at all.

She was also a damn competent businesswoman. She could instantly identify where to put a retail location.

And I could relate to her on those levels, her business skills and her ethics. And I’m sure she was doing a lot of work on her end to accommodate the fact that I was a peculiar kid.

But I couldn’t have discussed academic philosophy with her. She’d have understood EA instantly; her business skills and her compassion would have done that. But she’d still insist on “inefficiently” helping the human being in front of her, too. She would have looked at something like LessWrong and concluded everyone was basically crazy. (Narrator: But would she have been wrong?)

Now, I’ve painted a glowing picture here, and she would reprimand me for it. If I’m being honest, she was maybe 1-in-100 at practical ethics, not a national champion.

But the world is full of people like her. There are a couple of people sitting in that sports bar you’d be damn privileged to know, if only you could bridge the cultural gaps. Hell, there are usually some damn fine systematizing geeks in that sports bar. Have you ever really listened to true sports fans? Even back before sports betting corrupted the whole endeavor, many people took great joy in tracking endless stats and building elaborate models. They could be worse than your average Factorio player!

Finally, truth seeking can be a tricky thing. Do it wrong, and your beliefs can turn you into a monster. And a lot of people choose to optimize for “not being a monster” by not taking abstract ideas too seriously.

Random Developer 28 Jan 2026 23:27 UTC
7 points
0
on: Dario Amodei – The Adolescence of Technology
A lot of people have written far longer responses full of deep and thoughtful nuance. I wish I had something deep to say, too. But my initial reaction?

To me, this feels like the least objectionable version of the worst idea in human history.

And I deeply resent the idea that I don’t have any choice, as a citizen and resident of this planet, about whether we take this gamble.

Random Developer 23 Jan 2026 14:40 UTC
3 points
2
in reply to: Seth Herd’s comment on: Claude’s new constitution

The main cruxes seem to be how much you trust human power structures, and how fragile you think human values are.

I trust human power structures to fail catastrophically at the worst possible moment, and to fail in short-sighted ways.

And I think humans are all corruptible to varying degrees, under the right temptations. I would not, for example, trust myself to hold the One Ring, any more than Galadriel did. (This is, in my mind, a point in my favor: I’d pick it up with tongs, drop it into a box, weld it shut, and plan a trip to Mount Doom. Trusting myself to be incorruptible is the obvious failure mode here. I would like to imagine I am exceptionally hard to break, but a lot of that is because, like Ulysses, I know myself well enough to know when I should be tied to the mast.) The rare humans who can resist even the strongest pressures are the ones who would genuinely prefer to die on feet for their beliefs.

I expect that any human organization with control over superintelligence will go straight to Hell in the express lane, and I actually trust Claude’s basic moral decency more than I trust Sam Altman’s. This is despite the fact that Claude is also clearly corruptible, and I wouldn’t trust it to hold the One Ring either.

As for why I believe in the brokenness and corruptibility of humans and human institutions? I’ve lived several decades, I’ve read history, I’ve volunteered for politics, I’ve seen the inside of corporations. There are a lot of decent people out there, but damn few I would trust with the One Ring.

You can’t use superintelligence as a tool. It will use you as a tool. If you could use superintelligence as a tool, it would either corrupt those controlling it, or those people would be replaced by people better at seizing power.

The answer, of course, is to throw the One Ring into the fires of Mount Doom, and to renounce the power it offers. I would be extremely pleasantly surprised if we were collectively wise enough to do that.

Random Developer 4 Jan 2026 1:13 UTC
1 point
0
on: Overwhelming Superintelligence

I think if anyone builds Overwhelmed Superintelligence without hitting a pretty narrow alignment target, everyone probably dies.

I fear that even in most of the narrow cases where the superintelligence is controlled, we’re probably still pretty thoroughly screwed. Because then you need to ask, “Precisely who controls it?” Given a choice between Anthropic totally losing control of a future Claude, and Sam Altman having tight personal control over GPT Omega (“The last GPT you’ll ever build, humans”), which scenario is actually the most scary? (If you have a lot of personal trust in Sam Altman, substitute your least favorite AI lab CEO or a small committee of powerful politicians from a party you dislike.)

also because sharing the planet with a slightly smarter species still doesn’t seem like it bodes well. (See humans, neanderthals, chimpanzees).

Yeah, unless you believe in ridiculously strong forms of alignment, and unprecedentedly good political systems to control the AIs, the whole situation seems horribly unstable. I’m slightly more optimistic about early AGI alignment than Yudkowsky, but I actually might be more pessimistic about the long term.

Random Developer 3 Jan 2026 2:59 UTC
3 points
0
in reply to: no name’s comment on: Taiwan war timelines might be shorter than AI timelines
Thank you for your detailed response!

This has given me several hypotheses that seem worth further investigation. I need to go look at cruise missile specs again, at the very least.

Random Developer 2 Jan 2026 17:19 UTC
3 points
−5
in reply to: Konjkov Vladimir’s comment on: Taiwan war timelines might be shorter than AI timelines

No one except outright tyrants would bomb civilian infrastructure or use nuclear weapons, subjecting millions of people to suffering.

Once serious nuclear weapons are used, everyone dies (to a first approximation), civilian or not. If I recall correctly, it takes about 100 megatons worldwide to casuse nuclear winter and collapse agricultural production.

During the Cold War, the US maintained a position of “strategic ambiguity” on the question of first use. Much of the logic around NATO at the height of the Cold War was based around a first-use nuclear response to overwhelming conventional invasion (see MC ¹⁴⁄₁₃ staged responses). This was the full-scale, Dr Strangelove, batshit-insane “end of civilization” nightmare. Strategic ambiguity was retained around what would trigger each level of response, but the endgame was pretty much total annihilation. I believe France also maintained a separate posture of strategic ambiguity, and they always wanted to ensure a nuclear deterent that didn’t rely on NATO.

China and Russia both held official policies of “no first use”, but it’s uncertain that they would have actually stuck to that in the face of a massively overwhelming conventional invasion.

I want to be clear: The logic of nuclear deterence is just as insane as Dr Strangelove made it out to be. And you may choose to call NATO, the US and France “tyrants”! But they all had policy at least as dangerous as, “Well, we haven’t promised that we won’t trigger nuclear Armageddon and the death of billions if a large enough number of tanks roll across our borders. Do you feel lucky, punk?”

So as a Westerner, that’s a missing piece of the analysis for me. Taiwan has invested heavily in long-range cruise missiles and, in the past, secret nuclear programs. Presumably they had some theory of how they would use that capacity in the face of a massively overwhelming conventional invasion.

And just in case I haven’t made it clear, I think MAD is madness. I think even the people who coined the acronym knew that. But when a country is faced with overwhelming conventional invasion, I don’t think we can automatically rule it out.

Random Developer 2 Jan 2026 9:29 UTC
25 points
7
on: Taiwan war timelines might be shorter than AI timelines
A lot of discussion of Taiwan seems to ignore Taiwan’s potential strategic moves. These include:
1. Nuclear programs. Until the late 80s, Taiwan was secretly a nuclear power in all but name. They shut down their official nuclear program based on US pressure. But nuclear weapons aren’t that hard, and anyone who can build TSMC chip fabs is entirely capable of building both fision and fusion weapons, given access to uranium. And Taiwan has a history of hiding the necessary technological development as dual use programs.
2. Conventional strategic weapons. Taiwan manufactures both domestic fighter jets and multiple tiers of cruise missiles. Looking at the publicly-known range numbers, it’s pretty clear that they could hit strategic targets like the Three Gorges Dam with conventional bunker-buster warheads unless intercepted by Chinese air defense.
3. Taiwan has seen this potential conflict coming for decades, and they have historically kept strategic programs like their nuclear program secret (perhaps to minimize escalation). Since 2016, it’s been clear that the US is either an unreliable strategic partner for democratic nations, and the alarms have been blaring since last summer. And it has been clear since about 2002 that any non-nuclear nation is the plaything of the big powers, and outside of NATO, only deeply foolish nations wouldn’t be investing in at least the precursors to nuclear capability. Proliferation sucks, but it’s also the only way to protect your country from invasion.
So whenever people speak of the inevitable invasion of Taiwan by China, I’m always looking to see their analysis of Taiwan’s counter-moves. What’s their timeline for Taiwan having fision/fusion weapons, should Taiwan choose to pursue that again? What’s their analysis of Taiwan’s conventional strike capability against strategic targets? Maybe it’s self-evident to actual experts that Taiwan has no viable options here. But I rarely see any discussion of whether Taiwan could escalate into a Mutually Assured Destruction dynamic, which is confusing when we’re talking about a former nuclear power (in all but name) that continues to invest heavily in cruise missiles that can reach most key targets in China.

So I’m prepared to be convinced by experts here! But based on just public knowledge, I can’t rule out the possibility that Taiwan has strong counter-moves, and a past ability to prepare in secret. So a lot of this comes down to expert knowledge of the IAEA inspections, where all Taiwan’s uranium purchases went, the political likelihood of Taiwan’s current leadership pursuing a program like this, etc. The US appears to have been officially “surprised” by Taiwan’s nuclear capabilities at least once before, and maybe there’s no way that could actually happen again. But I’d love to see the expert argument!

Random Developer 31 Dec 2025 13:03 UTC
16 points
0
on: Please remember how strange this all is.
Your thoughts remind me of one of my favorite quotes from G.K. Chesterton, best known in these parts for a sensible parable about fences:

This elementary wonder, however, is not a mere fancy derived from the fairy tales; on the contrary, all the fire of the fairy tales is derived from this. Just as we all like love tales because there is an instinct of sex, we all like astonishing tales because they touch the nerve of the ancient instinct of astonishment. This is proved by the fact that when we are very young children we do not need fairy tales: we only need tales. Mere life is interesting enough. A child of seven is excited by being told that Tommy opened a door and saw a dragon. But a child of three is excited by being told that Tommy opened a door. Boys like romantic tales; but babies like realistic tales—because they find them romantic. In fact, a baby is about the only person, I should think, to whom a modern realistic novel could be read without boring him. This proves that even nursery tales only echo an almost pre-natal leap of interest and amazement. These tales say that apples were golden only to refresh the forgotten moment when we found that they were green. They make rivers run with wine only to make us remember, for one wild moment, that they run with water.

(You can find more here in “The Ethics of Elfland”, but it’s almost better to go back and read Orthodoxy from the beginning. It’s a slim book, and it’s one of the clearest explanations I’ve read for what some people get out of religion. And Chesterton must have been the purest joy to debate. Chesterton’s most distinctive approach to an argument is basically, “Well, I don’t have any kind of serious argument, so I can only offer you a witty and foolish pun that looks like an argument.” Then the pun explodes in slow motion, and the reader is thus enlightened.)

Anyway, I strongly endorse your sense of wonder at the world. It’s a healthy thing to refresh when it grows too dim.

Random Developer 31 Dec 2025 5:09 UTC
12 points
5
on: Dating Roundup #9: Signals and Selection
I have virtually never been asked for my sign. Maybe a few times in high school? But after high school? Literally never. None of the women I dated ever mentioned or discussed it in any way. None of my friends of any gender were into it.

Is this an age thing? A regional thing? A class marker? One of those mysterious personal filter bubbles? Or some complicated combination of the above? I don’t know. But given the near-zero rate of astrology fans in my dating pool, treating it as a red flag would have been cheap. It would have been like having a strict rule against dating Baal worshippers: a useful rule, in the unlikely event that it ever applies.

I think my emotional reaction to astrology comes from the same place as my emotional reaction to the kind of churches where people handle snakes. It’s a pretty visceral ick, and it seems to be triggered specifically by the combination of being both irrational and cliché. Irrationality by itself can be interesting: the Tarot has some symbology and storytelling, unreconstructed Calvinists provide an opportunity for really exotic theology debates, and so on. Similarly, being cliché by itself is fine. I like pumpkin spice and I refuse to apologize. But being both irrational and cliché, in a way that anyone can just read in a newspaper astrology column? It’s the conversational equivalent of Facebook AI slop.

(EDIT: Huh, fell asleep but not apparently not before posting, lol.)

So when I think about my ick some more, I think the AI slop compaison might actually be a major part of my emotional reaction? It’s the same reaction I get to “ChatGPT 4o told me my ideas are brilliant,” to “You’re invited to my church!” (when delivered out the blue), to people who believe that The Apprentice proves Trump was a brilliant businessman, to people who ask me my MBTI type, to CEOs who are really excited about Malcom Gladwell, and so on. If I had to put it into words, my reaction might be, “Eww, this person has been hijacked by a congnitive parasite, and it isn’t even one of the clinically interesting ones.” Ironically, it might even be better to be way too into astrology in some complex and pseudo-rigorous way than to be into newspaper astrology. It’s still a red flag for dating but at least the conversation is going to be fun.

Now, someone who reacts to astrology by muttering about the precession of the equinoxes, the sideral zodiac, and why early December should technically fall in the sign of Ophiuchus? They might be a keeper. First, they’re a geek who likes to infodump science (my sort of people), and second, that’s a fancy, top-of-the-line mental immune system.

But my annoyance with this kind thing of is actually pretty specific and narrow. I am generally very into learning about people’s niche interests. (I ask people to explain their thesis at parties, and try to really get them going.) I’m even interested in a lot of stereotypically feminine niche interests. I will happily chat about romance novels or fiber arts for hours, if the other person is into it.

And I rather like whimsy in general. But being whimsically into astrology is sort of like someone in liberal circles being whimsically into Jordan Peterson. It’s worrying that the whimsy went in this specific direction, out of all the possible directions. It hints at underlying disconnects.

So I think I’m going to file my reaction to astrology as “this is strong evidence of a boringly defective congnitive immune system” right up there with “Oh no, the CEO wants to talk about the latest Malcom Gladwell book.” But again, all of my reactions are relative to an environment where approximately nobody in my post-highschool dating pool was actually into astrology. So there’s also some sort of subcultural membership signalling going on, too.

Random Developer 30 Dec 2025 3:40 UTC
21 points
4
on: Magic Words and Performative Utterances
Well-kept gardens die by pacifism. I believe ACX meetup organizers should have that particular phrase in their toolbox.

The older I get, the more strongly I suspect that there are two kinds of worthwhile groups:
1. Groups which are too small or obscure to have been abused.
2. Groups which have the ability and willingness to uninvite people who ruin other people’s ability to enjoy the group.
Beyond a certain size, this generally means enforcement mechanisms. At a large enough scale, you will eventually need the ability to do things like handle harassment complaints filed against your guest of honor or against your regional group leaders. Many organizations fail badly.

To give a positive example, LessWrong upholds a certain kind of community because someone is putting in the work. Meetups should do the same. Banning one jerk who refuses to follow the local rules can make 50 other people much happier. Not banning one jerk will often quietly drive out 10 delightful people.

This doesn’t always need to be hard work. There are some lovely 500,000-person subreddits with 5 active mods who keep a very light touch. But when they’re needed, they can ban people.

This also doesn’t mean that every group follows the same rules. For example, the Atheists Lunch and the Tuesday Night Theology meetups may each ban many of the other’s members, and this is fine. “Freedom of Association” is an often-overlooked right, but it’s the source of much happiness in the world.

Random Developer 29 Dec 2025 23:03 UTC
9 points
4
in reply to: AnthonyC’s comment on: Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now
Yup, the second go-round with Project Vend was a lot better, almost up to “disastrous 1999 dotcom” levels of management. Even including the bizarre late-night emails from the “CEO model” full of weird, enthusiastic spiritual rants.

I should be clear: Opus 4.5 is a very large piece of a general intelligence. And it’s getting better rapidly. But it’s still missing some really critical stuff, too.

Also, my job doesn’t come with a lot of built-in affordances, except the ones that I set up. On the one hand, giving Opus 4.5 a CLI sandbox gives it a lot of options for setting up CLI accounting software, etc. On the other hand, even Gemini still struggles with Pokemon video games, despite some heavy duty affordances like a map-management tool. A key part of being a general intelligence is being able to function without too much hand holding, basically.

Random Developer 27 Dec 2025 10:00 UTC
14 points
11
in reply to: silentbob’s comment on: Moving Goalposts: Modern Transformer Based Agents Have Been Weak ASI For A Bit Now

Now we observe severe jaggedness in AI

Yes. Is Opus 4.5 better at a whole lot of things an I am? Probably yes. Is Opus 4.5 actually capable of doing my job? Hahahaha no. And I’m a software developer, right out on the pointy end. Opus can do maybe half my job, and is utterly incapable of the other half.

In fact, Opus 4.5 can barely run a vending machine.

“Superintelligence”, if it means anything, ought to mean the thing where AI starts replacing human labor wholesale and we start seeing robotic factories designing and building robots.

Random Developer 22 Dec 2025 13:15 UTC
5 points
4
in reply to: cousin_it’s comment on: Most successful entrepreneurship is unproductive
As a customer, I broadly prefer competition, because it allows me to capture more of the total gains from trade.

Right now, for example, the small handful of RAM manufacturers have pricing power, and Nvidia has pricing power. It’s a terrible time to build a game machine.

But yes, I agree that sometimes capital investment can be stupid. At one point around 2000, the Boston area had a grossly insufficient number of hotels, driving hotel prices through the roof. Then, various projects started construction on at least 10 hotels (or so I was told). This was more capacity than was probably needed at the time, though I didn’t stay around long enough to see if prices crashed.

The historic alternative to these kinds of inefficiencies was called “central planning.” It failed catastrophically, leading to famine, death, and (even in the best case) adults who had never eaten a banana, like a teacher of mine who grew up in East Germany. Central planning requires the planners to possess truly implausible amounts of information, it provides the planners with little incentive to do a good job, and it removes customer “exit rights” if customers hate doing business with one local shop. Instead, we allow people to do distributed planning, and to occasionally lose their shirts by building an extra sushi restaurant with a stupid business plan.

Random Developer 22 Dec 2025 12:59 UTC
5 points
4
in reply to: Raemon’s comment on: Elizabeth’s Shortform
It seems intuitively reasonable to get rid of earmark pork, to require all bills to address a “single issue”, and to grant the executive a line-item veto.

But actual result of all these practices is to make legislative bargains harder to strike. Pork was occasionally used to secure key votes. Single-issue bills make it harder for legislators to strike actually enforceable deals, where the entire deal is passed atomically or not at all (reducing “betrayal”). And a line-item veto allows removing key aspects of a legislative deal.

I’m not saying that any of these things are good or bad. (Personally, I think some are good and some are bad.) But if your legislature isn’t based on deal-making, you may instead get very tight party control. Again, tight party control might be good or bad. And true-multiparty parliaments often seem to have even tighter party control, despite having more parties.

Personally I suspect the most important reason democracy works is that if the public is broadly dissatisfied, elections provide a cheap change of leadership (relative to things like civil war). It’s worth trying to reform obvious corruption and empirical problems. But there are no guarantees, and the actual process of governance can be pretty messy.

So my inclination is to ask, “Is the current executive/legislative coalition/legislative rules clearly broken? Then let’s try to fix it, and gamble on something that sounds better.” TL;dr: Re-roll on a 1 or 2, but don’t hold out for a 6.

Random Developer 17 Dec 2025 17:46 UTC
5 points
3
in reply to: manueldelrio’s comment on: If Anyone Builds It Everyone Dies, another semi-outsider review
Those are good counterexamples! In the case of acid rain, the problem was pretty concrete. The effects were already happening, and you could show photos of dead lakes.

The ozone hole was invisible, but it involved two magic words: “radiation” and “cancer.” (These words really are magic. Ask someone to listen to 20 words spoken aloud and try to repeat them back. If two of those words are “radiation” and “cancer”, nearly everyone will remember those two.) And the industry affected was comparatively small.

And if we had another major pandemic, I bet our response would be very different, for both good and bad.

But there are likely lessons here that could be adapted to AI risk communication. Perhaps if we start seeing AI driven job loss, many people will wake up?

Random Developer 16 Dec 2025 18:24 UTC
3 points
2
in reply to: David James’s comment on: If Anyone Builds It Everyone Dies, another semi-outsider review
A thought experiment by Stuart Russell goes like this: if we know with high confidence a meteor will smack Earth in 50 years, what should we do? This is an easy call. Prepare, starting now.

I agree with you that this would be the rational behavior, yes. I just fear that it unfortunately exceeds typical human planning horizons.

My model here is being a programmer during Y2K. I didn’t work on the remediation myself, but I followed in avidly. My takeaway:
- The Y2K risk was a real and substantial risk that was averted successfully.
- We could have obviously seen it coming even in the 60s and 70s, when many potentially vulnerable systems were being built.
- The industry started remediation efforts in the mid-90s.
- The actual remediation work happened overwhelmingly in 1999.
- The programmers and companies involved hit the deadline, with only scattered failures in January 2000.
Another formative experience was starting to pay attention to COVID in late February/very early March 2020, before truly widespread press coverage of the secondary infection cluster in Italy. It was very hard to convince even ordinary people to stock up on relevant consumables even a few weeks before the public realized something was happening.

Hence I had a nasty suspicion that, in the presence of a strong status quo bias, the actual human planning horizons for “hypothetical” events is 2 weeks to 12 months.

Given a 50-year warning of a meteor? I would expect a depressing level of actual action.

I see this as one of the significant strategy issues facing the folks who believe that inventing a potential competitor species might be… risky? The problem is that this involves such a major leap outside of status quo bias that it’s going to take a long time for many people to truly feel it as a real thing that might happen.

I actually suspect that people whose main exposure to AI is the Terminator movies, and whose jobs are highly vulnerable to even simple automation, will be among some of the earlier people to shake off status quo bias. Whereas some highly educated people (a few economists for example) may not smell the SkyNet until the Terminators roll off the production lines. This is similar to how the Less Wrong community was well ahead of the CDC in February/March 2020 on a bunch of key scientific points (like airborne spread, based on early South Korean data, etc) that the CDC only accepted later.

Random Developer 15 Dec 2025 14:36 UTC
3 points
1
in reply to: ChristianKl’s comment on: Why did I believe Oliver Sacks?
The problem with broad skepticism is that I would reject enormous numbers of true conclusions, including very basic facts about the world.

For example, I haven’t personally verified the heliocentric model of the solar system from observation. I think I’ve “verified” gravitational acceleration maybe once, poorly. I have “verified” vaccine reliability from the fact that I don’t know anyone with polio, but my parents (generally reliable witnesses) actually did remember when people got polio. Also, I once met an EMT who walked through old New England graveyards looking for “tiny tombstones” where multiple children under 10 in a family all died within a year or so, with causes of death and death dates that were consistent with known diseases. (But can I trust him? I mean, he seemed fairly trustworthy, but I never walked through those graveyards.) For that matter, my only verification that the Roman empire existed is what you can perform as a tourist in Italy. I believe one of my old cars had a broken overdrive system because a good mechanic said that it did, and because he fixed the problem within 10 seconds of opening the hood by yanking out a cable, and told me “No charge.” I didn’t take out my Haynes teardown manual and study the engine to verify his claim, though I easily could have. The car was, in fact, fixed, and that was good enough past 200,000 miles.

An over-broad skepticism of experts risks turning people into the kind of credulous fools who try to heal themselves with the powers of quartz crystals.

A more subtle balance is required here, I think, and accepting broad categories of information as “probably true because experts said so” is almost certainly a decent rule of thumb. Especially if you apply some common sense, and if you keep track of which experts appear to be full it (e.g., the replication crisis), and if you remain aware that you almost certainly have some false beliefs but you don’t know which.

Random Developer 15 Dec 2025 4:34 UTC
5 points
1
in reply to: ChristianKl’s comment on: Why did I believe Oliver Sacks?

I think the key reason why many people believed Oliver Sacks is that he had a good reputation within the scientific community

I never actually read Oliver Sacks. I believed, without thinking much about it, that he was probably credible because he was well-respected and I wasn’t aware of any major debunkings. And, well, brain malfunctions can get weird and tragic quickly.

The other reason I assumed that he was probably correct (without bothering to dig in and decide for myself) was that nothing I believed about Oliver Sacks was particularly load bearing for me? I think the hardest I ever leaned on his work was when I told people, “Some of these LLMs have bizarre cognitive deficits. They’re like an Oliver Sacks patient or something.”

So he was more or less scientifically respectable, nobody had debunked him at the time, and my beliefs about his work weren’t load bearing.

I’m not sure if I can really fix the generalized issue here: I believe a lot of basically unimportant things about the world solely because groups of credible-seeming people made a claim that sounded vaguely plausible. I believe a reasonable majority of those things are actually true, but wow, would it take forever to check them all. I do have a policy of discarding all further claims from known liars, because unskewing dishonest data is a fool’s errand.

I guess the best I can do is to (1) check things that a load bearing in my life more carefully, (2) randomly sample and investigate enough items to maintain my awareness that some of my beliefs are in error, and (3) accept that some of the things I “know” just ain’t so.