Viliam

Karma: 25,573

Viliam 4 Sep 2025 20:34 UTC
14 points
0
on: Viliam’s Shortform
Why do AIs write using the “AI style”? (Overuse of words like “delve” or constructions like “it is not X, it is Y”.) I mean, they are trained on human texts, people call them “stochastic parrots”, so how is it possible for them to develop a distinct style? Isn’t that like the only thing they—practically by definition—shouldn’t do? I would expect them to write text that superficially seems like written by humans, but if you look closely, you see that the style is there but the substance is not. Instead, the style is wrong.
There are many things that I don’t understand about LLMs (practically all of them), but this one bothers me because I would expect the exact opposite.
(Is it possible that the “AI style” is actually a style of some category of humans that produce a lot of texts that for some mysterious reason most people never see, so those texts are over-represented in the learning data? Journalists maybe?)

Viliam 3 Sep 2025 20:47 UTC
4 points
0
on: Structural engineering in software engineering
I think this analogy nicely illustrates the short-term and long-term problems. Building the house without foundations means building it faster. So the guy who builds it will be considered a genius, and he will be happy to tell you about how doing useless things such as building the foundations is a waste of time. Then the guy collects a huge bonus and disappears to another new project, and you are left with the “trivial” task of fixing the cracks when they appear. Fixing the cracks turns out to be more expensive in long term than building the house… which may be interpreted as further evidence that you are less competent than your genius colleague.
My software development experience suggests that developers who make new projects prefer languages without static typing and code without unit tests, developers who maintain existing projects prefer languages with static typing and with unit tests.

Viliam 3 Sep 2025 20:27 UTC
2 points
0
on: Models vs beliefs
When things are complicated, we probably develop a partial model first (taking only some things into account) and maybe a full model later. The partial model may give worse results, compared to trusting the experts blindly. And yet, making your own model brings you closer to understanding how things work.
Like, maybe you are trying to estimate some X = A + B + C + D + E, but so far your model is only X = A + B + C.

Viliam 3 Sep 2025 14:31 UTC
0 points
−2
in reply to: sarahconstantin’s comment on: sarahconstantin’s Shortform
https://scottsumner.substack.com/p/the-myths-of-chinese-exceptionalism Scott Sumner says maybe stop using anti-China saber-rattling as an excuse to tell lies in the service of whatever your agenda happens to be?
In comments, I find it funny how people disagree on the meaning of “evidence”.
Scott: “All the so-called ‘evidence’ [for covid lab leak] has been refuted.”
...received a link to ACX article, with quoted “the short version is that they were doing lots of gain of function research [...] The market is unlikely to be the origin of the pandemic, because the original Lineage A strain wasn’t found there [...] The most likely explanation for all these facts is that WIV went ahead and did the gain-of-function research they said they were going to do”
Scott: “I don’t think you know the meaning of the term “evidence”. It’s doesn’t mean “speculation based on no facts.”″
As usual, Eliezer wrote something relevant a decade ago: “Scientific Evidence, Legal Evidence, Rational Evidence”.

Viliam 3 Sep 2025 12:58 UTC
2 points
0
in reply to: patrik-cihal’s comment on: patrik-cihal’s Shortform
I am not optimistic about civilization either. I mean, we have tons of wisdom available all around us (e.g. Wikipedia, pirated textbooks), people just don’t care, they prefer sharing conspiracy theories and pseudoscience on social networks. We have discovered quantum physics, but we still have theocracies that support religious terrorists. We obsess about cultural appropriation, while many people live in literal slavery. We can’t even rely on the fact that smart and civilized countries will outcompete the stupid and uncivilized ones, because it turns out that the latter can simply buy or steal the technology, and the former are incapable of making all their citizens smart, so they are always potentially one election away from a disaster.
Of course AIs have a potential to make it much worse. But the problem is, in current situation, we can’t even coordinate on stopping them. As weapons, they are far more dangerous than nukes, because at least the nukes are not dual use, so there is no profit motive for getting halfway there.
I guess my point is that we should not see civilization and AIs as alternatives. It’s rather, the AIs are so dangerous because the civilization is so fucked up that we don’t have much of a chance to develop the AIs in a safe way.

Viliam 2 Sep 2025 12:37 UTC
4 points
0
in reply to: johnswentworth’s comment on: Dating Roundup #7: Back to Basics
If that is what people mean when they talk about “the dance of flirtatious mutual escalation”, I wish they would describe it accurately—it’s a dance of “one person escalates and the other passively approves”.
There are various dance styles, they probably meant this one.

Viliam 2 Sep 2025 12:29 UTC
3 points
0
on: Dating Roundup #7: Back to Basics
As far as I know, men tend to choose female friends who are attractive.
you should focus on making platonic friends with people you find attractive, without trying to date them at all, and then see what happens?
If the situation is symmetric (i.e. women also tend to choose male friends who are attractive), then if you are not attractive, what happens is that your attempts to make platonic friends will probably not be reciprocated.
[testing mind-reading] also has a clear negative selection effect for ‘men who go around hitting on everyone a lot’
Not sure about this. A man who stretches his attention thin between many women may be worse at reading any particular woman’s mind, but better at noticing the statistical patterns. (He says the right things on the third date not because he understands her perfectly, but because he knows from experience that saying the same thing on the second date is usually too early and saying it on the fourth date is usually too late.)
I think that if you select for smooth players, then smooth players is exactly what you will get. Although you will select against clumsy players; the type of guy who barely has any skill beyond “approach a lot”.
Also, this selects for men who are familiar with your (sub)culture, which seems like a good thing. Unless you grew up in a culture that you hate, of course, then it seems like a bad thing.
One detailed complaint from a client who paid $4,500 articulated the perceived unfairness of being matched with men who had only paid the $99 database fee, a critical detail she claims was never disclosed during the sales process.
This is hilarious.

Viliam 2 Sep 2025 10:49 UTC
3 points
3
in reply to: patrik-cihal’s comment on: patrik-cihal’s Shortform
I am sure there were articles written about this a decade ago. The short version is that if an AI wants to be friendly and still leave humans some meaning, it could e.g. solve the most pressing problems (such as death and war) and then let humans figure out the rest. Analogically to a parent who lets his children learn things, but gives them food, and makes sure they won’t get hurt.
One possible solution (not necessarily the only one), imagine being immortal, and if you get too sick AI will magically cure you, and it will prevent you from being a victim of crime, but otherwise everything is left for humans to do. You won’t starve, but the only way to eat a cake is to bake one, or pay another human to do it. (Maybe all generative AIs are banned?) Now it’s up to humans to learn science and invent things. I think that might be enjoyable.

Viliam 1 Sep 2025 8:33 UTC
4 points
2
on: Should we align AI with maternal instinct?
Thank you for writing this.
(Generally, I think that approaches like “we need to use this applause light to align the AI” are mistaken.)

Viliam 1 Sep 2025 8:14 UTC
5 points
0
in reply to: asher’s comment on: asher’s Shortform
Ah, an avalanche of thoughts that apply to heuristics in general...
- yes, it is also my experience that this is a useful heuristics
- but there are exceptions
- and when this becomes widely known, of course the bad actors will adapt and say the right things
The most useful example for this heuristics is when people say things like “everyone is selfish” etc. For example:
Brent promoted a cynical version of the world, in which humans were inherently bad, selfish, and engaged in constant deception of themselves and others. He taught that people make all their choices for hidden reasons: men, mostly to get sex; women, mostly to get resources for their children. Whenever someone outside Black Lotus made a choice, Brent would dissect it to reveal the hidden selfish motivations. Whenever someone outside Black Lotus was well-respected or popular, Brent would point out (or make up) ways that they were exploiting those weaker than them for their own benefit. This became a self-fulfilling prophecy. One interviewee identified this ideology as the worst harm of Black Lotus, more than sexual boundary violations or being coerced into taking drugs. It took a long time to rebuild their ability to trust and have faith in other people.
The opposite extreme is naive people who assume that everyone is trying their best, and if there is a dysfunction somewhere, it must be because somehow no one has ever approached those people and told them “uhm, could you please do the right thing?”. If we do that, certainly the problems will get fixed! (Similar pattern: naive religious people who assume that others are atheists simply because they have never heard of this book called Bible.)
Problem is, sometimes people happen to be surrounded by a statistically unrepresentative sample of population. Young people may overgeneralize examples from their family, rather than from themselves. I suspect that there are many kids who e.g. happen to have an abusive father, so they generalize “all men are abusive”, because (1) they have no idea how it works in other families, and it makes sense to assume that all other families also keep secret about what happens at home, and (2) it is too painful to admit that it’s your family that happens to suck.
One thing that I hate about this heuristics is that it discourages learning. Like, suppose that you start like the naive kind of person who assumes that everyone always tries their best to help others. Then you get burned, because you happen to meet a predator who specializes on this kind of victims. Afterwards you try to share your lesson with other people in a similar situation… but they incorrectly apply this heuristics to you and conclude that you are the bad guy. (Incorrectly: you say “some people are evil”, not “all people are evil”. But to them it sounds similar.)
Then there is the usual thing about bad actors being anti-inductive. As soon as it becomes known that good people assume the best and bad people assume the worst, smart scammers will adapt their stories, and will weaponize this heuristics against anyone who suspects them or just advises caution in general. (Just like they have already adapted to heuristics such as “sincere people will look in your eyes, the insincere will avoid looking at you because they feel ashamed” by taking care to look straight in your eyes when they are trying to scam you.)
...and despite all objections that I just made, it is a useful heuristics (as long as you remember the exceptions).

Viliam 1 Sep 2025 7:03 UTC
4 points
2
in reply to: the gears to ascension’s comment on: the gears to ascenscion’s Shortform
dunno, I see two confused opinions, maybe if you explained what exactly is the part that made you interested.

Viliam 29 Aug 2025 23:32 UTC
2 points
0
in reply to: Gabriel Alfour’s comment on: The Gabian History of Mathematics
I am not an expert, but it seems to me that you still need some kind of intuition to guide you, considering that it is impossible to define even such seemingly simple ideas like “natural number” in first-order logic.
Machine proofs are a great thing, but without interpretation, we wouldn’t know what they are actually about. Like, the machine can show that if you have axiom X, Y, Z, you can prove Q. Sure, but what do those axioms mean, and why should I care about them? (I mean, I could just generate random axioms, and let the machine find some statements that can be proved from them, but would anyone want to read that?)

Viliam 29 Aug 2025 23:03 UTC
4 points
0
in reply to: eggsyntax’s comment on: eggsyntax’s Shortform
Use the following prompt to evaluate it: Please take a look at the attached project and provide a careful critical analysis of it from a scientific perspective. Start with a 200-word summary of the project...
Have you actually tried doing that, e.g. with the https://www.lesswrong.com/moderation#rejected-posts ?

Viliam 29 Aug 2025 22:50 UTC
4 points
0
in reply to: Elizabeth’s comment on: Ben Pace’s Shortform Feed
dollar, magician hat, sleepy face, no division by zero, plus sign, ugly face, uhhh no idea what is this one, wizard, bug under looking glass, check your thoughts, cocaine, broken chain, unbroken chain, delta (I thought we already had this one)

Viliam 29 Aug 2025 22:34 UTC
4 points
0
in reply to: Charlie Steiner’s comment on: Wikipedia, but written by AIs
What I had in mind was that it would avoid the “AI psychosis” situation, where whatever crackpot theory you believe, the AI agrees to it. You may believe that the quantum recursion is the source of human consciousness, but the encyclopedia will not support that.
There is a risk of supporting popular misconceptions, e.g. horoscopes. Whether that actually happens, and whether it can be prevented, is something I would like to see as an experiment.

Wikipedia, but written by AIs

Viliam29 Aug 2025 16:37 UTC

31 points

9 comments4 min readLW link

Viliam 28 Aug 2025 16:35 UTC
5 points
1
in reply to: mruwnik’s comment on: Gamblification
I actually have Scrum Master training… so I can confidently say that most companies that claim to do Scrum are actually closer to doing its opposite.
- retrospective—either not doing it at all because it is a waste of time (yeah, remove the only part of the process where the developers can provide feedback about the process, and then be surprised that the process sucks), or they are doing some perverted reverse-retrospective where instead of developers giving feedback to the company, it is company giving feedback to developers, usually that they should work faster and make fewer mistakes
- there is no “Jira” in the Scrum Guide
- there is no “Confluence” in the Scrum Guide
- and the retrospective is exactly the place where the developers should say “Jira and Confluence suck, we want some tools that actually work instead”, but I already mentioned the retrospective
- there are no managers in Scrum; and no, it is not about renaming the manager’s role, but about giving the team autonomy
- the only deadlines in Scrum are the ones negotiated at sprint planning, and not in the sense of “management says that the deadline is after two sprints, but you have the freedom to choose which part you implement during the first sprint, and which part during the second one”
I only experienced something like actual Scrum once, when a small department in a larger company decided to actually try Scrum by the textbook. It was fun while it lasted. Ironically, we had to stop because the entire company decided to switch to “Scrum”, and we were told to do it “properly”. (That meant no more wasting time on retrospectives; sprints need to be two weeks long because more sprints = more productivity; etc.)

Viliam 28 Aug 2025 16:28 UTC
2 points
0
in reply to: mruwnik’s comment on: Thane Ruthenis’s Shortform
Maybe the sufficient reason is that LLMs learn both on the bad code and the good code. And on the old code, which was written before some powerful features were introduced to the language or the libraries, so it doesn’t use them. Also, they learn on automatically generated code, if someone commits that to the repository.

Viliam 28 Aug 2025 12:35 UTC
7 points
1
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
I use Claude for some hobby coding at home. I find it very useful in the following situations:
- I am too lazy to study the API, just give me the code that loads the image from the file or whatever
- how would you do this? suggest a few alternatives (often its preferred one is not the one I choose)
- tell me more about this topic
So it saves me a lot of time reading the documentation, and sometimes it suggests a cool trick I probably wouldn’t find otherwise. (For example, in Python list comprehension, you can create a “local variable” by iterating through a list containing one item.)
On the other hand, if I let Claude write code, it is often much longer code that when I think about the problem, and only let Claude give me short pieces of code.
Maybe I am using it the wrong way. But that too is a kind of risk: having Claude conveniently write the code for me encourages me to produce a lot of code because it is cheap. But as Dijkstra famously said, it is not “lines produced” but “lines spent”.

Viliam 28 Aug 2025 12:12 UTC
2 points
0
in reply to: Aprillion’s comment on: Gamblification
so, the end of “agile”?

Viliam

Wikipe­dia, but writ­ten by AIs

Wikipedia, but written by AIs