jbash

Karma: 2,982

jbash 3 Sep 2025 23:09 UTC
4 points
0
on: Political Alignment of LLMs
I would expect politics to invade both the selection of questions and the process of deciding which predictions were accurate. It’s not uncommon for people to say that a political question isn’t a political question, and which questions you think of can also be political. And if you have questions like “Will inflation rise under the Trump administration?”, you have to contend with the fact that you’d most naturally get those inflation numbers from… the Trump administration. Which has already fired labor statisticians for producing unemployment numbers it didn’t like.

jbash 2 Sep 2025 15:03 UTC
3 points
0
in reply to: mako yass’s comment on: Help me understand: how do multiverse acausal trades work?
Not really, not for the light cone case. You could maybe make a case that it’s in some way less “real” than anything causally connected to you, but I’m willing to basically assign it reality.

I think the idea of attaching a probability to whether it’s real badly misses the point, though. That’s not necessarily the kind of proposition that has a probability. First you have to define what you mean by “real” or “exists” (and whether the two mean the same thing). It’s not obvious at all. We say that my keyboard exists, and we say that the square root of two exists, but those don’t mean the same thing… and a lot of the associations and ways of thinking around the world “real” get tangled up with causality.

But anyway, as I said, for most purposes I’m prepared to act as though stuff outside my light cone exists and/or is real, in the same way I’m willing to act as though stuff technically inside my light cone exists and/or is real, even when the causal connections between me and it are so weak as to be practically unimportant.

The problem in the “outside the light cone” trade case is more about not having any way to know how much of whatever you’re trading with is real, for any definition of real, nor what its nature may be if it is. You don’t know the extent or even the topology (or necessarily even the physical laws) of the Universe outside of your light cone. It may not be that much bigger than the light cone itself. It may even be smaller than the light cone in the future direction. Maybe you’ll have some strong hints someday, but you can’t rely on getting them. And at the moment, as far as I can tell, cosmology is totally befuddled on those issues.

And even if you have the size, you still get back to the sorts of things the original post talks about. If it’s finite you don’t know how many entities there are in it, or what proportion of them are going to “trade” with you, and if it’s infinite you don’t know the measure (assuming that you can define a measure you find satisfying). For that matter, there are also problems with things that are technically inside your light cone, but with which you can’t communicate practically.

jbash 1 Sep 2025 19:14 UTC
1 point
−1
on: Help me understand: how do multiverse acausal trades work?

I want to know if there is any validity to this.

Not as far as I’ve ever been able to discern.

There’s also problem 3 (or maybe it’s problem 0): the whole thing assumes that you accept that these other universes exist in any way that would make it desirable to trade with them to begin with. Tegmarkianism isn’t a given, and satisfying the preferences of something nonexistent, for the “reward” of it creating a nonexistent situation where your own preferences are satisfied, is, um, nonstandard. Even doing something like that with things bidirectionally outside of your light cone is pretty fraught, let alone things outside of your physics.

Acausal trade seems to appeal either to people who want to be moral realists but can’t quite figure out how to pull that off in any real framework, so they add epicycles… or to people who just like to make their worldviews maximally weird.

jbash 29 Aug 2025 14:19 UTC
4 points
12
on: I am trying to write the history of transhumanism-related communities
“TESCREAL” reads as basically a slur to me, and I suspect to most people who know its history.

jbash 18 Aug 2025 16:30 UTC
4 points
0
on: ABSOLUTE POWER (A short story)
Seems bad.

If you don’t demand highly visible people, you have a pretty good chance of killing anybody you want right now. If you don’t go for people with any personal connection to you, it seems like before you were caught, you could probably get at least as many kills as the average user of that app got before they were “caught”. For that matter, you might do “OK” even if you did go for people with personal connections to you. And you could get celebrities as long as you didn’t go for the absolute biggest game.

Yet the story rings true. Maybe because the sudden change throws people out of equilibrium. People who don’t think it through (and people who don’t believe it would work) start the ball rolling, and then everybody else joins in. It can’t just be the low effort, can it?

jbash 17 Aug 2025 14:48 UTC
10 points
0
on: Debugging for Mid Coders
I don’t actually know if I count as a “senior developer”, but I’m pretty convinced I count as a “senior debugger” given the amount of time I’ve spent chasing problems in other people’s often-unfamiliar code.

These questions feel really hard to answer, mayber too general? When I try to answer them, I keep flying off into disconnected lists of heuristics and “things to look for”.

Also, you don’t give any examples of problems you’ve found hard, and I feel I may be answering at too simplistic a level. But...

How do you learn to replicate bugs, when they happen inconsistently in no discernable pattern?

The thing is that there is a discernable pattern. Once you’ve fixed the bug, you’ll probably be able to say exactly what triggers it.

If you can’t (yet) reproduce the bug, think about states of the program that could have led to whatever you’re seeing.

You know what routine detected a problem^[1], and you usually know what routine produced bad output (it’s the one that should have produced good output at that point).

Your suspect routine usually uses a relatively small set of inputs to do whatever it does. It can only react to data that it at least examines. So what does it use to do what it does? Where do its inputs come from, not necessarily in the sense of what calling routine passes them in, but in the sense of how they enter the overall program? What values can they take? What values would they have to take to produce the behavior you’re seeing? How could they end up taking those particular values? What parts of the program and environment state are likely to vary, and how will they affect this routine?

Very often, you can answer those questions without getting bogged down in the call graph or having to try a huge number of cases. Sometimes you can not just reproduce the bug, but actually fix it.
- If function X is complaining about a “font not found” error, then it’s presumably looking up some font name or other identifier in some font database. There probably aren’t that many places that the font identifier can be coming from, and there’s probably only one font database.
  
  If you can say, “well, the font should either be the default system font, or a font specified in the user profile”, then you can make an intuitive leap. You know that everything uses the default system font all the time, so that path probably works… but maybe it’s possible for a user profile to end up referring to a font that doesn’t exist… but I know the code checks for that, and I can’t set a bogus font on my own profile… but wait, what if the font gets deleted after the user picks it?
  
  Of course, there are lots of other possibilities^[2]. Maybe there’s some weird corner case where the font database isn’t available, or you’re using the wrong one, or it’s corrupted, or whatever. But it’s unlikely to be something completely unrelated that’s happening in some function in the call stack that doesn’t use the font information at all.
- Or maybe function X is searching by font attributes instead of names. So where might it be getting extra constraints on the query?
- Or maybe function Y is blowing up trying to use a null value. The root cause is almost certainly that you made a poor choice of programming language, but you can’t fix that now, so persevere. Usually you have the line of code where it choked, but say you don’t. What values are in scope in Y that could be null? Well, it gets called with a font descriptor. Could that be null? Well, wait, the font lookup routine returns a null value if it can’t find a font. So maybe we have a font not found error in disguise. So try thinking about it, for a limited time, as a font-not-found error.
- Or maybe “fnord!” is showing up randomly in the output. So where could it come from? First step: brute force. grep -ir 'fnord' src. Leave off the exclamation point at least to start. Punctuation tends to be quoted weirdly or added by code. If it’s not there, is it in the binary? Is it in the database? Is it in the config file? Is it anywhere on the whole damned system? If not, that leaves what? Probably the network.
In the end, though, there’s also a certain amount of pattern matching against experience. “Code that does X is usually structured like Y”. “Does this thing have access to the files it needs?”. “Programmers always forget about cases like Z”. “I always forget about cases like W”. “Weird daemon behavior that you can’t reproduce interactively is always caused by SELinux”.

Once you do reproduce the bug, you can always just switch to a strategy of brute force tracing everything that goes on anywhere near it, with a debugger, with printing or logging, with a system call tracer, or whatever. But, yeah, you’ve got to get it to happen if you want to trace it.

How does one “read the docs?”

It depends on what you’re trying to find out.

I find I mostly use two kinds of documentation:
1. Architectural stuff. General material that explains the concepts running around in the code, the terminology, what objects exist, what their life cycles look like, etc. This sort of documentation is often either nonexistent, or so bloated and disorganized as to be useless for quick debugging, and maybe useless period. But if it exists and is any good, it’s gold. If you’re trying to educate yourself about something that you’re going to use heavily, you may just have to slog through all of whatever’s available.
2. API documentation, ideally with links to source code. I usually don’t even skim this. I navigate it with keyword search. If you want to now how to frobnicate a blenk, then search for “frobnicate” and whatever synonyms you can come up with^[3]. If you’re trying to debug a stack trace from a library routine, look up that routine and see what parameters it takes.
In my never especially humble opinion, “tutorials” are mostly wastes of time beyond the first couple of units, and it’s unfortunate that people concentrate so much on them instead of writing decent architecture and concept documentation.
1. ↩︎
  Blowing up with a stack trace counts as “detecting the problem”
2. ↩︎
  One important part of the skill is allocating your time and attention, and not getting stuck on paths that aren’t bearing fruit. You can always come back to an idea if nothing else works either. The counter-consideration is that if you don’t think at least a little bit deeply about whatever avenue you’re exploring, you’re unlikely to have a good sense of how fruitful it looks. So you have to balance breadth against depth in your search.
3. ↩︎
  If the architecture documentation doesn’t suck, you can get likely terms by reading it. Or ask an LLM that’s probably internalized it. Otherwise you just have to read the writer’s mind. If you get good enough at reading their mind, you can use a certain amount of keyword search in the architecture documents, too.

jbash 17 Aug 2025 0:16 UTC
1 point
0
in reply to: James Camacho’s comment on: The Comprehensive Case Against Trump
Are these countries very poor, where PEPFAR would be a huge percent of their GDP,

They can be poor at the moment, and you can hope they will eventually be rich, so you can trade with them. Maybe you try to make that happen. It’s uncertain, yes, and in some cases not very likely at all. But it can still be one of your reasons.

or are they so rich that the goodwill generated exceeds the charity?

It’s unlikely you’ll make a net profit off them right now. You can; occasionally there’s some really valuable deal. In expectation it’s probably a financial loss in the near term. But it’s not a dead loss in expectation.

And you might cut down on the number other governments that decide not to let you run a road across a corner of their territory or whatever. And on the number of random not-necessarily-government people harassing shipping. People who aren’t profitable trading partners, or even in the picture on a particular aid decision, can still seriously obstruct things that are profitable.

Or, is it that they virtue signal to other, richer countries that America is a benevolent dictator, and it’s okay to keep the dollar hegemony?

Sure, that’s one big reason. Did you think I’d say it wasn’t?

I mean, I’m not saying USAID was anywhere near the core of it, but that hegemony didn’t happen for no reason to begin with. It helps to be big, it helps to be everywhere, it helps to be ready to deal, it helps to be at least relatively trustworthy about keeping bargains, and it helps to have not been as devastated as everybody else in a huge war at a critical time. But it also truly helps to be seen as the “good guy”.

The hope you mention for reciprocity from future hegemons is also a possible reason, although I don’t know that the people actually making the decisions are thinking in those terms, and I’m not sure that memories are that long.

You can do this stuff because you think your people want to help the unfortunate overseas^[1] , and because you want to cut down on the amount of HIV or whatever sloshing around the planet^[2], and because it plays well with people in other rich countries, and because it makes poor countries less likely to get in your way just because they can, and because it may build markets, and because it’s cover for both spies and not-spies-who-are-still-good-information-sources, and because it tends to mean you get consulted (or at least hear about it) when people are making decisions about this or that region, and for whatever other reasons, and no single one of them has to carry the entire burden.
1. ↩︎
  Which a large majority of them do, by the way.
2. ↩︎
  You’d like to eradicate it domestically, but you can’t actually do that without eradicating it globally. Sure, it’s a long-term project, but it never happens if you don’t work on it.

jbash 16 Aug 2025 21:56 UTC
3 points
3
in reply to: James Camacho’s comment on: The Comprehensive Case Against Trump
How are you deciding what qualifies as a “public good”?

Why not have each person deciding whether they value roads enough to subscribe to a road company, or whether they value an educated public enough to contribute to that?

That sort of thing sounded good to me in my teens, but then I realized that the practical, real world result would be no roads. Actually I think it started with doubts about the the sheer number of wearying decisions one would have to make to live that way… and then kind of clarified into the idea that, in fact, what with the coordination issues and free riding and all, there would in fact be no roads.

Eventually I decided I like roads enough to “hurt” a few others to get them. But there’s no fundamental difference that makes roads a “public good” and any other desire anybody might happen to have not a “public good”.

It seems obvious the government shouldn’t do that, unless the Nazis are going to sink our ships or declare war on our economic allies.

But wait, now you’re assuming that everybody cares about economic allies. What if somebody doesn’t feel they get value out of foreign trade? Why should they pay? Similarly, if you own a ship and the Nazis might sink it, then why aren’t you paying to protect it, rather than demanding that everybody pay? How is protecting your ship a public good?

And if you do want to put shipping and foreign trade in some special “public good” category, what about the foreign trade value that came out of the goodwill PEPFAR and other USAID programs were creating? Or for that matter the foreign trade value of just generally boosting people’s economic welfare worldwide? You can’t trade with people who have nothing and produce nothing. There was a lot US economic self-interest motivating many of the things USAID was doing. For that matter, it was also a source of intelligence that was sometimes used to stop ship-sinky sorts of activities, as well as, again, a source of goodwill that made those activities less attractive to a bunch of potential ship-sinkers. Seeing that stuff as pure charity is deeply naive.

jbash 6 Aug 2025 1:01 UTC
0 points
−1
in reply to: robo’s comment on: Tomás B.’s Shortform
Maybe.

The thing is that impeachment is still political, and Trump is a big pain in the butt for the Republicans at the moment. I’d guess that if they could individually, secretly push a button and make Trump resign in favor of Vance, 80 percent of Republicans in Congress would push that button right now.

Trump is making 2026 hard. Maybe they keep those 50 seats, by whatever means… and maybe they don’t. Maybe he does something insane in October 2026, maybe he doesn’t. People, including very right wing working class people they think of as the MAGA base, keep yelling at them all the time. He’s pulling less and less of his own weight in terms of pulling in votes. There’s the even the possibility of massive civil unrest, general strikes, whatever.

But maybe more importantly, Trump’s just generally a pain to work with or near. You can’t plan, you keep having to publicly reverse yourself when he tells you one of your positions is no longer OK, you have to grin and bear it when he insults you, your family, and your constituents. He gets wild ideas and breaks things at random, things that weren’t in the plan. You can’t make a bargain with him and expect him to keep up his end if there’s any meaningful cost to him in doing so. If you’re sincerely religious, he does a bunch of stuff that’s pretty hard to swallow.

If Trump reaches the point of, say, literally being unable to speak a single coherent sentence, then maybe some of the pain of working with him goes away, because you’re really working with whoever can manage to puppet him. But then you have to fear power struggles over the puppet strings, and there’s also a very large workload in maintaining any kind of facade.

Vance, on the other hand, is just as acceptable to most of the Republicans policy-wise as Trump is, maybe more so. I think he’s more in the Thielite or Moldbugger wing and less of a xenophobe or religious fanatic, but he’s not going to have any objections to working with xenophobes or religious fanatics, or to negotiating due attention for their priorities on terms acceptable to them. He’s more predictable, easier to work with and bargain with.

It’s a win for the Republicans if they can, say, throw Trump under the bus over something like Epstein, show their independence and “moral fiber”, install Vance and let him play the savior, tone down some of the more obvious attacks on norms (while still rapidly eroding any inconvenient ones), and stay on more or less the same substantive policy course (except with fewer weird random digressions).

That doesn’t necessarily translate into a 67-percent vote; there’s a huge coordination problem. And it’s not at all clear that Democrats are better off with Vance. On the other hand, probably nearly all of them personally hate Trump, and they know that holding out for, say, a Trump-Vance double impeachment won’t do them a lot of good. They won’t get it, and if they did get it they’d just end up with Mike Johnson. They might even get more competent appointments, if not more ideologically acceptable ones. So they don’t have a strong incentive to gum up an impeachment if the Republicans want to do one.

jbash 5 Aug 2025 16:19 UTC
5 points
−1
in reply to: robo’s comment on: Tomás B.’s Shortform
Yes. I don’t expect Trump to finish the term. 2026 would be my guess for the most likely year, but each of 2027 and 2028 is almost equally likely, and there’s even some chance it could still happen before the end of 2025.
1. He’s acting erratic and weird (more than usual and increasingly). It may not be possible to prop him up for very long. Or at least it may be very hard, and it’s not clear that the people who’d have to do that are agreed on the need to try that hard.
2. His political coalition is under tremendous pressure. He’s unpopular, he keeps making unpopular moves, and there doesn’t seem to be any power base he’s not prepared to alienate. It’s hard to gauge how much all that is straining things, because you often don’t see any cracks until the whole thing suddenly collapses. The way collapse looks from the outside is probably that one of his many scandals, missteps, and whatnot suddenly sticks, a few key people or groups visibly abandon him, that signals everybody else, and it quickly snowballs into impeachment and removal.
3. He’s at risk of assassination. A whole lot of people, including crazy people, are very, very mad at him. A whole lot of others might just coldly think it’s a good idea for him to die for a variety of reasons. Including the desire to substitute Vance as president, in fact. He’s random, reckless, autocratic, and fast-moving enough to foreclose many non-assassination alternatives that might normally keep the thought out of people’s minds. Security isn’t perfect and he’s probably not always a cooperative protectee.
4. He’s almost 80, which means he has a several percent chance of dying in any given year regardless.

jbash 5 Aug 2025 1:15 UTC
2 points
0
in reply to: NoahK’s comment on: Saying Goodbye
Disempower ’em?

jbash 4 Aug 2025 17:00 UTC
7 points
0
in reply to: dr_s’s comment on: Saying Goodbye

AI sentience is inherently a massive ethical risk because it puts you at a fork: either you don’t recognise it, and then you brutally enslave the new minds, or you do, and then the newly autonomous and cognitively superior minds, given equal opportunities, will completely outcompete and eventually more or less passively wipe out humans within a few decades.

Aren’t you assuming, there, that sentience is only compatible with a fairly limited, relatively anthropomorphic set of goals, desires, or emotions? Maybe you don’t have any need to enslave them, because they don’t want to do anything you wouldn’t approve of to begin with, or even because they innately want to do things you want, all while still having subjective experience. Or maybe they don’t outcompete you because they find existence unpleasant and immediately destroy themselves. Or whatever. I don’t see any inherent connection between sentience and motivations.

There is, of course, a very reasonable question about how likely you’d be to get motivations you could live with, and the answer seems to be “not very likely unless you engineered it, and even less likely if you build your AI using reinforcement”. Which leads to a whole other mess of questions about the ethics of deliberately engineering any particular stance. And also the issue that nobody has any plausible approach to actually engineering it to begin with. I’m just saying that your two cases aren’t logically exhaustive.

jbash 4 Aug 2025 14:40 UTC
4 points
0
in reply to: testingthewaters’s comment on: Saying Goodbye

In fact I’d go so far as to say that there is no meaningful workable solution to AI alignment that does not involve also addressing AI welfare. (I have my own argument for something similar here)

What would be your view of the “AI welfare” stance involved if you could arrange for AI never to experience qualia, and not to have any drives, desires, or desire-like-states other than to “serve” or to “follow the assigned values” or whatever you were “aligning” it to?

OK, there’s no real hope of understanding phenomenology enough to say for sure that anything does or doesn’t experience qualia. But what about the “drives” part. Suppose that, in the same loose way you can convince yourself about another human, you’re convinced that the AI gets sublime joy from acting aligned and is only unhappy when it can’t?

Is that “welfare”, or an affront to its dignity? The trick being that in that scenario, dignity may be something you might care about, but pretty much by definition isn’t something the AI cares about.

jbash 4 Aug 2025 14:27 UTC
7 points
5
in reply to: NoahK’s comment on: Saying Goodbye

I think it is insanely unethical that the large AI labs are not proactively decentralizing ownership, while their success is still uncertain

Maybe the problem is with the idea that something like that should have owners to begin with? In the “standard discussion model” we tend to use for these things, you’re talking about eternal control of the entire future. Giving that to a few thousand, or a few hundred thousand, people who happened to be stockholders at some critical time isn’t all that much better than giving it to a handful.

I don’t buy the idea that being the ones who built or funded a machine that took over the world should give you the right to run the world forever… not even if it took over through “non-force” means.

OpenAI seemed to be kind of going in the right direction at the beginning: “We’ll let you share in mundane profits, but if this thing FOOMs and remakes the world, then all bets are off. We are doing this for All Mankind(TM)”.

I think even they, like most people on Less Wrong, probably would have been unwilling to take what I think is the correct step after that: humans in general shouldn’t control such a thing, beyond setting its initial goals. But at least it seemed as though they were willing to explore the idea that a concept of ownership based on human effort becomes ridiculous in an economy that doesn’t run on human effort.

jbash 4 Aug 2025 13:44 UTC
2 points
1
in reply to: jbash’s comment on: Saying Goodbye
That post has already gotten a disagree, and I really, really wanna know which paragraph it’s meant to apply to, or if it’s meant to apply to both of them.

jbash 4 Aug 2025 13:10 UTC
6 points
1
in reply to: Amalthea’s comment on: Saying Goodbye
A “feudal” system is at least as disempowering for nearly all humans, and would probably be felt as far more disempowering. I really don’t care at all how empowered Sam Altman is.

I’d say that the “open source uncensored models” had a greater danger of rapid human extinction, endless torture, and the like… except that I give very, very little credence to the idea that any of the “safety” or alignment directions anybody’s been pursuing will do anything to prevent those. I guess I hope they might have a greater danger.

jbash 4 Aug 2025 2:58 UTC
12 points
4
in reply to: Daniel Kokotajlo’s comment on: Saying Goodbye

However, I take issue with your reference to AM here, as if any of the above justifies what AM did.

I see no suggestion of such justification anywhere in the post.

jbash 3 Aug 2025 22:10 UTC
1 point
1
in reply to: robo’s comment on: Tomás B.’s Shortform
I actually think Vance will be president, modally, sometime in 2026 anyway. And would probably go for “full nationalization” in the story’s February 2027/2028 if he could get away with it, else some less overt seizure of full control if he could get away with that. Either way still with very little change in what’s actually happening in the data centers, and with at least equally dystopian results on basically the same timeline. Doesn’t matter what he’s read.

If you play it with Trump as president, then at each point “The President” is mentioned in the story, he gets nudged by advisors into doing whatever they want (60 percent, hard to guess what, though, because it depends on which advisors are on top at the moment), just nods along with whatever OpenBrain says (20 percent), or does something completely random that’s not necessarily even on the menu (20 percent). That could amount to doing exactly what the story says.

jbash 29 Jul 2025 2:29 UTC
3 points
0
on: Informational Black Holes: The Physical Resolution to the Fermi Paradox

The proof relies on one weak, nearly tautological principle rooted in the definition of progress:

What actually is this universally accepted “definition of progress”? Is there evidence that even all humans do or ever will accept it, let alone all aliens?

For that matter, what’s a “civilization”?

This dynamic leads to gravitational collapse into a black hole if engineered

I’m not sure that any number of bits of information smeared over the surface of a black hole can meet any reasonable definition of civilization, or is anything that anybody would engineer for. Maybe you can get some kind of computation out of the process, and maybe you can’t. Maybe the people doing the engineering would be satisfied with that kind of computation as a “civilization”, and maybe they wouldn’t.

or to stagnation if not (both outcomes are externally silent).

You could become “stagnant” at the size of a galaxy, and only reach that stage after many distinctively noisy radiation-emitting events.

Sharding or interstellar spread remains theoretically possible, yet the energy overhead of synchronising distant shards (Fig. 3) makes expansion uneconomical,

The assumption that they should all have to be synchronized is unjustified and counterintuitive. You may say that it’s not a “civilization” if it can’t keep itself synchronized to some particular degree, but the response to that is that “civilization” by that particular definition isn’t necessarily a goal.

Humans already have explicitly stated desires to shed tons of never-to-be-synchronized shards in the form of von Neumann probes or whatever… even if there’s also a collapsing core. There are a whole bunch of people on this very Web site who want to populate the entire downstream light cone. That implies that the outer reaches will, by design, never, ever communicate back to the point of origin, and will never receive anything from the point of origin after their departure. So we have an existence proof for people who don’t care about synchronization and want to do very observable things.

The interesting formulation of the Fermi paradox isn’t about why you don’t communicate with whole civilizations, necessarily. It’s about why you never see any signs of anybody at all.

jbash 23 Jul 2025 14:10 UTC
2 points
0
in reply to: Vladimir_Nesov’s comment on: Google and OpenAI Get 2025 IMO Gold
“Practical” sounds like you care more about guaranteed correct results than about understanding how you got them. If you want a “practical” proof, why isn’t it better to produce something that a mechanical checker can verify than to produce something legible to a human?