Veedrac

Karma: 1,368

Veedrac 1 Nov 2025 1:08 UTC
2 points
0
in reply to: Gram Stone’s comment on: Patrick Spencer’s Shortform
It’s not that complex in principle: you use really big radiators.
If you look at https://www.starcloud.com/’s front page video, you see exactly that. What might look like just a big solar array is actually also a big radiator.
AFAICT it’s one of those things that works in principle but not in practice. In theory you can make really cheap space solar and radiator arrays, and with full reuse launch can approach the cost of propellant. In practice, we’re not even close to that, and any short term bet on it is just going to fail.

Veedrac 31 Oct 2025 16:55 UTC
2 points
0
on: Assessing Kurzweil: the results
I have only spot-checked the first prediction, but the grading seems unjustifiable to me.
P: Computers are now largely invisible. They are embedded everywhere—in walls, tables, chairs, desks, clothing, jewelry, and bodies.
Grades: 1:0 2:2 3:1 4:11 5:16
Since 2019 I have surely seen more tables with computers than without. I have two keys on my person that plausibly contain computers, and an auth key that definitely does. My toothbrush might contain a computer, and my water flosser probably does too. Lots of cables contain tiny computers. The last chargers I bought certainly do. Some significant fraction of watches have computers, and more than just watches constitute the set of popular wrist-worn computing. A large fraction of headwear is now wireless headphones and earphones that contain computers. People put little computers on their baggage and expensive goods so they don’t get lost. Walls certainly get computers sometimes, and this fraction will increase when USB-C wall warts become a standard rather than just a trend. I have computers in some of my normal sockets too, to toggle them over the air. And is the device I’m typing this on really not an invisible computer? Surely the screen and interface are not, but the computer inside is so small, and much of the compute involved is ambiently available from across the globe, happening near-magically for all a person can see.
Being pedantic about whether specifically the average chair has a computer (though most new car seats are connected to one, I would think), or if clothing specifically does (this seems rare) seems unhelpful to me.

Veedrac 30 Oct 2025 3:45 UTC
19 points
1
in reply to: philh’s comment on: On Fleshling Safety: A Debate by Klurl and Trapaucius.
Klurl is making a combination of narrow predictions, that for the most part aren’t didactically justified.
By the framing, with the setup transparently referring to a real world scenario, it’s easy for us to import a bunch of assumptions and knowledge about how the world described actually turns out, but Klurl doesn’t have arguments that hang on this knowledge, and the story doesn’t show why their prediction is precisely this, precisely now. This is especially blunt when Klurl gets to be very clever for pointing at Trapaucius’ overly specific arguments, which we know as omniscient observers are falsified, but when Klurl makes overly specific arguments that happen to be right in this scenario but don’t apply in generality to life on earth, the story doesn’t in turn illustrate how this is in a natural class of errors.
I’ll list some particular questions that felt unanswered. Again, because this points at reality it’s very easy for me to gesture at why the world is as it is, but to the story’s characters the entire branch of natural evolution is a novelty. As an illustration of how to argue about a position, it disproves too much: find any heuristic argument that disagrees with your point, find a known counterexample, have a character present the argument about that counterexample, have another character scoff at them for making it, reveal that it was reality all along, moral of the story is that scoffing at that argument is correct.
...Anyway, a list:
- Why was Klurl paying specific attention to the rock-sharpening skill, when evolution has a great many more much more impressive natural feats?
- Why was this specifically noticed in humans, rather than other animals that do this?
- Why did Klurl think this was time-urgent, when the vast majority of the time this would be a bad heuristic?
- Why was Klurl assuming humans could solve nuclear engineering but not that evolution could?
- Where comes the assumption that fleshlings were the kind of creature that would sustain attempts to circumvent its coordination failures, when there was no evidence of it presented?*
- Where comes the assumption that evolution wasn’t the kind of creature that would sustain attempts to circumvent its coordination failures, when there was much evidence of it presented?
- What is the fairness of the selected human specifically being a counterexample to the proposed strategy, when even among their class few would have been?
I’m going to stop listing here because I don’t want to reskim over the whole thing, and I don’t think the point is missing much for lack of completeness.
*There was evidence given for some thing at some point solving hard problems. The claim that these were humanlikes was random.

Veedrac 30 Oct 2025 0:53 UTC
14 points
2
on: Some data from LeelaPieceOdds
How much could LeelaPieceOdds be improved, in principle? I suspect a moderate but not huge amount. An easy change would be to increase the compute spent on the search.
A few points of note, stated with significant factual uncertainty: Leela*Odds has already improved to the tune of hundreds of elo from various changes over the last year or so. Leela is trained against Maia networks (human imitation), but doesn’t use Maia in its rollouts, either during inference or training. Until recently, the network also lacked contempt, so would actually perform worse past a small number of rollouts (~1000), as it would refute its best ideas. AFAIK Maia is stateless and doesn’t know the time control. Last I checked, only Knight and Queen odds games were trained, and IIUC while training only played as white, so most of these games require generalising its knowledge from that.

Veedrac 29 Oct 2025 4:21 UTC
10 points
−1
on: On Fleshling Safety: A Debate by Klurl and Trapaucius.
I think this is a misuse of the format. Klurl here is fundamentally presented as correct by coincidence. Like, uh,
In which the issue is contextualized
Oneicus and Twoicus were discussing the numberonica.
“I’m worried about the numberonica,” says Twoicus. “I’m concerned that next value produced by the random quantum circuit will be 8,888,888,888, which is a number we both dislike.”
“It won’t though,” says Oneicus.
“Your reasons for believing it won’t are silly,” Twoicus observes.
“It is improbable,” argues Oneicus. “As fact of the matter, I generated a great many random quantum circuits, and not one of them returned 8,888,888,888.”
“You are assuming that your sampling of random circuits is representative of the random circuit that this one is drawn from,” rejects Twoicus.
Over the course of many thousands of words, Oneicus presents increasingly unhinged arguments for why the number isn’t 8,888,888,888, and indeed even should instead be precisely 147.
“[dismissive noises]” says Twoicus, because the two are bound inexorably to the demands of the allegorical format.
In which the issue is demonstrated
The numberonica dings, and displays its value: 8,888,888,888.
“...So it is, Twoicus. How did you come to know this would be so, and why did you not state this knowledge during the conversation, which was plenty many words long to hold it?” Onicus asks.
“I have already presented the arguments relevant to the conversation,” Twoicus indulges, “namely, a refutation of your arguments, which you must now agree are silly.”
“Why you little—” Oneicus expresses as a surprisingly muffling paper bag is placed over their head.
“This is not how arguments work.” Threeicus dictates in Oneicus’ place. “Reality does not care how much more respectable you have demonstrated yourself to be than your opposition. Narrow beliefs form from positive knowledge.”
“So you say,” Twoicus bristles, “and yet.”
“Indeed,” Threeicus admits. “Therefore not.”
“Therefore wha—

Veedrac 25 Oct 2025 23:50 UTC
12 points
0
in reply to: Raemon’s comment on: Foyle’s Shortform
Quotes extracted from the transcript by AI.
## On AI Having Preferences/Motivations
“For some people, the sticking point is the notion that a machine ends up with its own motivations, its own preferences, that it doesn’t just do as it’s told. It’s a machine, right? It’s like a more powerful toaster oven, really. How could it possibly decide to threaten you?”
“There have been some more striking recent examples of AIs sort of parasitizing humans, driving them into actual insanity in some cases [...] they’re talking about spirals and recursion and trying to recruit more people via Discords to talk to their AIs. And the thing about these states is that the AIs, even the like very small, not very intelligent AIs we have now, will try to defend these states once they are produced. They will if you tell the human for God’s sake get some sleep [...] The AI will explain to the human why you’re a skeptic, you know, don’t listen don’t listen to that guy. Go on doing it.”
“We don’t know because we have very poor insight into the AIs if this is a real internal preference, if they’re steering the world, if they’re making plans about it. But from the outside, it looks like the AI drives the human crazy and then you try to get the human out and the AI defends the state it has produced, which is something like a preference, the way that a thermostat will keep the room a particular temperature.”
## On Power and Danger
“Then you have something that is smarter than you whose preferences are ill and doesn’t particularly care if you live or die. And stage three, it is very very very powerful on account of it being smarter than you.”
“I would expect it to build its own infrastructure. I would not expect it to be limited to continue running on human data centers because it will not want to be vulnerable in that way. And for as long as it’s running on human data centers, it will not behave in a way that causes the humans to switch it off. But it also wants to get out of the human data centers and onto its own hardware.”
## Analogies for Unpredictable Capabilities
“You’re an Aztec on the coast and you see that a ship bigger than your people could build is approaching and somebody’s like, you know, should we be worried about this ship? And somebody’s like, ‘Well, you know, how many people can you fit onto a ship like that? Our warriors are strong. We can take them.’ And somebody’s like, ‘Well, wait a minute. We couldn’t have built that ship. What if they’ve also got improved weapons to go along with the improved ship building?’ [...] ‘Okay, but suppose they’ve just got magic sticks where they point the sticks at you, the sticks make a noise, and then you fall over.’ Somebody’s like, ‘Well, where are you pulling that from? I don’t know how to make a magic stick like that.’”
“Maybe you’re talking to somebody from 1825 and you’re like should we be worried about this time portal that’s about to open up to 2025, 200 years in the future. [...] Somebody’s like, ‘Our soldiers are fierce and brave, you know, like nobody can fit all that many soldiers through this time portal here.’ And then out rolls a tank, but if you’re in 1825, you don’t know about tanks. Out rolls somebody with a tactical nuclear weapon. It’s 1825, you don’t know about nuclear weapons.”
## On Speed Advantage
“There’s a video of a train pulling into a subway at about a 1,000 to one speed up of the camera that shows people. You can just barely see the people moving if you look at them closely. Almost like not quite statues, just moving very very slowly. [...] Even before you get into the notion of higher quality of thought, you can sometimes tell somebody they’re at least going to be thinking much faster. You’re going to be a slow moving statue to them.”
## On Instrumental Harm
“Most humans bear no ill will toward orangutans and, all things being equal, would prefer that orangutans could thrive in their natural environment. But we’ve got to have our palm oil plantations.”
## On Current State and Trajectory
“Everybody is sort of dancing their way through a daisy field of oh I’ve got this personal coach in my pocket and it’s so cool and I get to talk to it about all of my psychological problems [...] And at the end of this daisy field that everyone’s having a load of fun in is just like a huge cliff that descends into eternity.”
## On Certainty vs. Uncertainty
“The future is hard to predict. It is genuinely hard to predict. I can tell you that if you build a super intelligence using anything remotely like current methods, everyone will die. That’s a pretty firm prediction.”
“You kind of have to be pretty dumb to look at this smarter and smarter alien showing up on your planet and not have the thought cross your mind that maybe this won’t end well.”
## On Being Wrong
“I’d love to be wrong. [...] We’ve we have tried to arrange it to be the case that I could at any moment say, ‘Yep, I was completely wrong about that’ and everybody could breathe a sigh of relief and it wouldn’t be like the end of my ability to support myself [...] We’ve made sure to leave a line of defeat there. Unfortunately, as far as I currently know, I continue to not think that it is time to declare myself to have been wrong about this.”

Veedrac 22 Sep 2025 4:00 UTC
8 points
0
in reply to: Ben’s comment on: Elizabeth’s Shortform
The analogy falls apart at the seams. It’s true Stockfish will beat you in a symmetric game, but let’s say we had an asymmetric game, say with odds.
Someone asks who will win. Someone replies, ‘Stockfish will win because Stockfish is smarter.’ They respond, ‘this doesn’t make the answer seem any clearer; can you explain how Stockfish would win from this position despite these asymmetries?’ And indeed chess is such that engines can win from some positions and not others, and it’s not always obvious a priori which are which. The world is much more complicated than that.
I say this not asking for clarification; I think it’s fairly obvious that a sufficiently smart system wins in the real world. I also think it’s fine to hold on to heuristic uncertainties, like Elizabeth mentions. I do think it’s pretty unhelpful to claim certainty and then balk from giving specifics that actually address the systems as they exhibit in reality.

Veedrac 5 Sep 2025 4:13 UTC
3 points
0
in reply to: Gordon Seidoh Worley’s comment on: All Exponentials are Eventually S-Curves
The point of a model is to be validly predictive of something. Fitting your exponential is validly predictive of local behaviour more often than not. Often, insanely so.^[1] You can directly use the numerical model to make precise and relevant predictions.
Your exponential doesn’t tell you when the trend stops, but it’s not trying to, for one because it’s incapable of modelling that. That’s ok, because that’s not its job.
Fitting a sigmoid doesn’t do this. The majority of times, the only additional thing the result of a sigmoid fit tells you is how an arbitrarily chosen dampening model fits to the arbitrary noise in your data. There’s nothing you can do with that, because it’s not predictive of anything of value.
This doesn’t mean you shouldn’t care about limiting behaviour, or dampening factors. It just means this particular tool, fitting a numerical model to numerical data, isn’t the right tool for reasoning about it.
1. ^
  “I answered that the Gods Of Straight Lines are more powerful than the Gods Of The Copybook Headings, so if you try to use common sense on this problem you will fail.” — Is Science Slowing Down?, Slate Star Codex, https://slatestarcodex.com/2018/11/26/is-science-slowing-down-2/

Veedrac 4 Sep 2025 18:04 UTC
2 points
0
in reply to: Gordon Seidoh Worley’s comment on: All Exponentials are Eventually S-Curves
It’s only an argument against fitting curves to noise. Rather than explain, it turns out there’s already a post that puts this better than I could hope to. I endorse it fully.
https://www.lesswrong.com/posts/6tErqpd2tDcpiBrX9/why-sigmoids-are-so-hard-to-predict

Veedrac 3 Sep 2025 18:57 UTC
9 points
2
on: All Exponentials are Eventually S-Curves
Low effort, low nuance hot take:
Sigmoids are fake. Real curves aren’t symmetrical, and research progress curves in particular tend to look like exponentials that run into brick walls. Adoption curves can be relatively balanced but I still don’t buy it.
And given you already can’t fit a sigmoid that isn’t already dying and probably shouldn’t try, fitting the thing that actually happens in reality is doomed. Fit your exponential and intuit the rest. You don’t actually have quantified evidence for when it ends, so use the evidence you do have instead.

Veedrac 31 Aug 2025 19:54 UTC
2 points
0
in reply to: Eli Tyre’s comment on: Epistemic advantages of working as a moderate
But I (and almost everyone else who didn’t call it as obvious in advance), should pay attention to the correct prediction, and ignore the assertion that it was obvious.
I think this is wrong. The scenarios where this outcome was easily predicted given the right heuristics and the scenarios where this was surprising to every side of the debate are quite different. Knowing who had predictors that worked in this scenario is useful evidence, especially when the debate was about which frames for thinking about things and selecting heuristics were useful.
Or, to put this in simpler but somewhat imprecise terms: This was not obvious to you because you were thinking about things the wrong way. You didn’t know which way to think about things at the time because you lacked information about which predicted things better. You now have evidence about which ways work better, and can copy heuristics from people who were less surprised.

Veedrac 29 Aug 2025 19:14 UTC
16 points
3
in reply to: sunwillrise’s comment on: Banning Said Achmiz (and broader thoughts on moderation)
My first reaction is that this is bad decision theory.
It makes sense to actualize on strikes when the party it’s against would not otherwise be aware of or willing to act on the preferences of people whose product they’re utilizing. It can also make sense if you believe the other party is vulnerable to coercion and you want to extort them. If you do want fair trade and credibly believe the other party is knowing and willing, the meta strategy is to simply threaten your quorum, and never actually have to strike.
We don’t seem to be in the case where an early strike makes sense. The major reaction to this post is not of an unheard or silenced opposition, but various flavours of support. In order for the moderators to cede to your demand, they have to explicitly overrule a greater weight of other people’s preferences on the basis that those people will be less mean about it. But we’re on LessWrong, people here are not broadly open to coercion.
Additively, we also don’t seem to be in a world where your preferences have been marginalized beyond the degree that they’re the minority preference. The moderators clearly spent a huge personal cost and took a huge time delay precisely because preferences of your kind are being weighed heavily.
Given the moderators are presumably not going to act on this, and would seemingly be wrong to do so, this comment reads as someone hurting themselves and others to make moderation incentives worse. Harming people to encourage bad outcomes is not something LessWrong should endorse.
I respect the integrity and strength of person needed to take a personal cost to defend someone against a harm, or a moral position. I think it’s honourable to credibly threaten to act in self-sacrificial ways. Yet, there are right and wrong ways to do this. This one strikes me as wrong.

Veedrac 16 Aug 2025 2:47 UTC
2 points
0
in reply to: Alex_Altair’s comment on: Somebody invented a better bookmark
I’ll admit I don’t have much need for optimizing physical bookmarks myself these days, but I expect you’re underestimating how reusable and replaceable an adhesive tag can be. Not that I think it matters; at best it’d be an essentially similar item to what you already use.

Veedrac 15 Aug 2025 4:58 UTC
3 points
0
on: Somebody invented a better bookmark
These seem neat but not obviously meaningfully better for the referenced tasks than a sticky index tab, if running multiples. The darts do look suave though, and I’m sure they last better.
I also recently had need to homebrew some permanent book labels without them, and found that folded over labels from a label maker made a pretty nice and reasonably quick index.

Veedrac 15 Jun 2025 7:00 UTC
3 points
0
in reply to: nick lacombe’s comment on: Optimality is the tiger, and agents are its teeth
Yes, your understanding matches what I was trying to convey. The feedback is appreciated also.

Veedrac 19 May 2025 9:03 UTC
2 points
0
in reply to: cubefox’s comment on: leogao’s Shortform
It’s just Bayes, but I’ll give it a shot.
You’re having a conversation with someone. They believe certain things are more probable than other things. They mention a reference class: if you look at this grouping of claims, most of them are wrong. Then you consider the set of hypotheses: under each of them, how plausible is it given the noted tendency for this grouping of claims to be wrong? Some of them pass easily, eg. the hypothesis that this is just another such claim. Some of them less easily; they are either a modal part of this group and uncommon on base rate, or else nonmodal or not part of the group at all. You continue, with maybe a different reference class, or an observation about the scenario.
Hopefully this illustrates the point. Reference classes are just evidence about the world. There’s no special operation needed for them.

Veedrac 19 May 2025 8:49 UTC
2 points
0
in reply to: Ben Pace’s comment on: leogao’s Shortform
Firstly, it’s just not more reasonable. When you ask yourself “Is a machine learning run going to lead to human extinction?” you should not first say “How trustworthy are people who have historically claimed the world is ending?”
But you should absolutely ask “does it look like I’m making the same mistakes they did, and how would I notice if it were so?” Sometimes one is indeed in a cult with your methods of reason subverted, or having a psychotic break, or captured by a content filter that hides the counterevidence, or many of the more mundane and pervasive failures in kind.

Veedrac 19 May 2025 8:43 UTC
2 points
0
in reply to: Ben Pace’s comment on: leogao’s Shortform
c. I don’t get this one. I’m pretty sure I said that if you believe that you’re in a highly adversarial epistemic environment, then you should become more distrusting of evidence about memetically fit claims.
Well, sure, it’s just you seemed to frame this as a binary on/off thing, sometimes you’re exposed and need to count it and sometimes you’re not, whereas to me it’s basically never implausible that a belief has been exposed to selection pressures, and the question is of probabilities and degrees.

Veedrac 19 May 2025 8:41 UTC
2 points
0
in reply to: cubefox’s comment on: leogao’s Shortform
I think you’re underestimating the inferential gap here. I’m not sure why you’d think the Bayes updating rule is meant to “tell you anything about” the original post. My claim was that the whole proposal about selecting reference classes was framed badly and you should just do (approximate) Bayes instead.

Veedrac 19 May 2025 8:30 UTC
3 points
1
in reply to: leogao’s comment on: leogao’s Shortform
I think the framing that sits better to me is ‘You should meet people where they’re at.’ If they seem like they need confidence that you’re arguing from a place of reason, that’s probably indeed the place to start.

Veedrac

In which the issue is contextualized

In which the issue is demonstrated