TsviBT

Karma: 11,169

TsviBT 21 Mar 2026 0:02 UTC
5 points
2
in reply to: Ruby’s comment on: Ruby’s Shortform Feed
But aren’t a lot of your tasks the sort of thing where
- there is in fact a ton of training-available data demonstrating good performance
- it’s cheap to experiment
- etc., other relevant peculiarities of your use cases
?

I think the claim might be true but I don’t see a super compelling reason to think so at the moment.

“Reasoning” helping with self-driving cars might be a compelling demo, but what it would be compelling about is “you can slap together robotics, big data for a specific domain, and some LLM reasoning stuff to duct tape some more of the decision-making, and get something that’s practically useful”. Generalizing to other robotics could kick off a revolution, but it would be slow-going I think?

There could be a fair amount of science overhang, where you just have to search hard enough to put X and needs-X together. E.g. people curing themselves by searching hard using LLMs. Exciting, but not an industrial revolution? In the grand scheme of science it’s not mostly that. A lot of the coolest stuff is really hard, which means there’s not that many people at the forefront, which means that people at the forefront are already familiar with a lot of what’s relevant.

If you can find domains where iteration can be done pretty automatedly, but it’s expensive enough that decision-making still matters, but decision-making is very cognitively costly, but getting kinda-okay-not-creative decision-making would still be quantitatively better, then you could unlock some sort of new paradigm of invention / discovery. E.g. automated labs running automated experiments designing proteins by gippity-tweaking, or similar. Like PACE. But that would also be hard to get started on.

What are other reasons to think this? Plausible I just haven’t seen the idea, haven’t tried too hard.

TsviBT 20 Mar 2026 19:12 UTC
2 points
0
in reply to: neo’s comment on: neo’s Shortform

1-on-1 communication as a significant bottleneck.

I think it’s one of the bottlenecks inter alia. What happens with two people with strong opinions / lots of knowledge / different backgrounds is that they fail to converge at all, often even on reference / ontology / communication, let alone actual facts / propositions / principles / plans. Because of this, there’s often no “ultimate recourse”; there’s ~no amount of research and thinking and problem solving that you can do in your head, such that you can then go out and spread the truth by just saying “take me to your leader” and then convincing the leader with facts and logic.

I also think the concept slightly falls apart when the users are not already knowledgeable / quick thinking / good at rigorous communication

Plausible, though I’m genuinely unsure, and would guess not. That is, I’d guess there are several kinds of uses for a hyperphone, and some of them would work across various gaps. For example, it’s plausible to me that speaking through a hyperphone would be my preferred way both of teaching and of learning something.

I’d guess there would be a steep learning curve.

There’d be a very steep learning curve at the beginning of development. It’s plausible that you could work things out so that there’s not that much learning curve after development. E.g. async text conversations are quite confusing if you’re stuck in the framework of normal live speech, and they do add some important kinds of friction, but mostly people get along.

Software wise, I think I’m aiming closer to better debates and debate tools, mainly because these things could be made public-facing and seem like an obvious use case for even present-day LLMs.

Possibly. One issue would be that debates are a bit more tense / higher stakes / constrained on average, I think, compared to 1 on 1 collaborative convos. For that reason, specifically for iterating on software, hyperphone convos might be easier to develop.

If you have any further thoughts regarding implementation of those things, I’d be eager to know. I’ll be trying to make my plans more specific.

If someone gets deep into this, I’d be happy to be one volunteer consultant among others about directions / ideas / considerations.

One strong recommendation I’d make is that, if you’re somewhat serious about it, GET DATA ASAP. E.g. set up some debates. About anything, between anyone who wants to debate. See what happens and see if there’s something truly hopeful there for you / the world, and then try to nourish that hopeworthy something. I wouldn’t begrudge anyone the a priori dreaming, which is more fun and can also be a very important ingredient in making cool stuff, but my guess would be that it wouldn’t go anywhere unless you feel hungry for data. You have to ask Thinking what it needs, not tell Thinking what it needs.

TsviBT 20 Mar 2026 7:27 UTC
2 points
0
in reply to: neo’s comment on: neo’s Shortform
Cf. https://tsvibt.blogspot.com/2023/01/hyperphone.html Could be nearby to some key software for interfacing deeply between minds. Someone might be able to code up a reasonable version in a few weeks using gippities, though that could fall prey to lock-in to wrong architecture choices. (I imagine, in particular, that handling audio in nonstandard ways might be an important early architectural choice. But maybe it’s fine.)

TsviBT 19 Mar 2026 0:51 UTC
18 points
0
in reply to: Kaarel’s comment on: testingthewaters’s Shortform
the real membrane-bound organelles were the friends we made along the way

TsviBT 18 Mar 2026 18:33 UTC
7 points
2
in reply to: Zach Stein-Perlman’s comment on: Zach Stein-Perlman’s Shortform
Cf. https://tsvibt.blogspot.com/2024/09/the-moral-obligation-not-to-be-eaten.html

TsviBT 17 Mar 2026 0:22 UTC
LW: 10 AF: 5
7
AF
in reply to: Thomas Larsen’s comment on: Thomas Larsen’s Shortform

AIs need to worry about their own alignment problem, meaning that they may not be able to self improve in an unconstrained fashion.

I haven’t thought too deeply about this, but I would guess that the AI self-alignment problem is quite a lot easier than the human AI-alignment problem.

Cf. https://www.lesswrong.com/posts/dho4JQytfHWXtTvkt/on-the-adolescence-of-technology?commentId=t2hKmhsS6yLyJFQwh

TsviBT 16 Mar 2026 8:23 UTC
8 points
2
in reply to: Mo Putera’s comment on: Less Dead
Not that much. I don’t understand what the actual experiment is, and in particular how much it is showing that we got algorithms (even opaquely) out of a fruit fly connectome. In any case, IIUC by all descriptions, it doesn’t really hit at the challenge of simulating neurons well enough that the simulated brain could learn anything. That’s what I expect to be difficult and to be needed for really useful simulations. (Separately I think brain emulation is dangerous; see https://www.lesswrong.com/posts/jTiSWHKAtnyA723LE/overview-of-strong-human-intelligence-amplification-methods#Problems )

TsviBT 16 Mar 2026 7:04 UTC
4 points
0
in reply to: Isaac King’s comment on: Less Dead
(I chatted with Aurelia some weeks ago and was overall pretty impressed, but she also made a claim like this about the fruit fly and I was somewhat skeptical / worried that the claim was overstated.)

TsviBT 14 Mar 2026 3:18 UTC
11 points
6
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
And perhaps effectively banning LLMs would cause a chilling effect much greater than if they were merely a research bet that didn’t work out. It would be a signal that AGI research is something the world does not want to see in any form, turning funding and talent away from the whole endeavor.
This is something I have some hope in (cf. https://www.lesswrong.com/posts/uBs6RJYtQbxZCRxSk/adam_scholl-s-shortform?commentId=fz5N4xn7Lema7RQJ9 ). A model that I got from @Davidmanheim (though I may have misunderstood) is that part of how “this is supposed to go” is not that there’s some big climactic treaty that does the final ban; but rather that there’s a treaty that bans something, with the stated intent of preventing the creation of AGI. Then they continue banning more stuff that’s in line with that intent, as it becomes clear that they should have banned it / meant to ban it. Or something like that.

TsviBT 12 Mar 2026 9:20 UTC
4 points
0
in reply to: cubefox’s comment on: TsviBT’s Shortform
(I don’t mind / would prefer this clarification in the comments?)

TsviBT 12 Mar 2026 9:19 UTC
21 points
12
in reply to: the gears to ascension’s comment on: adam_scholl’s Shortform
Agree qualitatively (and possible quantitatively). However, there’s a quite large knock-on effect, which is a strong bundle of signals of “AGI is bad, don’t make AGI”. These signals move in various directions between different entities, carrying various messages, but they generally push against AGI. (E.g. signaling legitimacy of the Stop position; the US signaling to other states; society signaling to would-be capabilities researchers; Congress self-signaling “we’re trying to ban this whole thing and will continue to add patches to ban dangerous stuff”; etc.)
What links here?
- TsviBT's comment on Thane Ruthenis’s Shortform by Thane Ruthenis (14 Mar 2026 3:18 UTC; 11 points)

TsviBT 11 Mar 2026 18:18 UTC
3 points
0
in reply to: cubefox’s comment on: TsviBT’s Shortform
In order to set out to believe falsely.

TsviBT 11 Mar 2026 6:44 UTC
2 points
1
in reply to: Lorxus’s comment on: Promoting enmity and bad vibes around AI safety
https://www.lesswrong.com/posts/Ht4JZtxngKwuQ7cDC/tsvibt-s-shortform?commentId=WsrmFhuysbmJ3xiPm

TsviBT 11 Mar 2026 6:43 UTC
26 points
11
on: TsviBT’s Shortform
In order to believe X falsely, one has to construct a plausible-ish world where X is the case. This distorted-world construction can happen piecemeal, in a sort of auto-sum-threshold attack.

In other words, suppose you want to rationalize your belief in X. You could simply abdicate all logic, and assert X while also asserting everything else that comes from your ordinary truth-seeking beliefs. (Well, that’s not simple, because what do we mean by believing in X, then? Something about actions? I’ll leave this as an open question here.)

However, that method is kinda hard sometimes. For one thing, if you are trying to truth-seek somewhat, you might follow the rule of believing logical implications of your beliefs. That could be out of pure ethics, or it could be for instrumental reasons. So you would very quickly run into contradictions. For another thing, you’d look silly, and have a hard time mind-melding with people on X-related things.

Better might be to construct a “logical counterfactual” where X is the case. You construct a plausible world nearest to our own, but where X is the case. That world may be logically / conceptually / strategically incoherent, but it could be less incoherent.

X is false, and X is related to many things that you know about. So there are many logical rends to mend.

This can happen piecemeal. You don’t have to construct the whole X world in one go. Indeed, doing so would draw a lot of attention, and in particular self-attention. If you try to mend a bunch of rends all in one go, where they all relate to your believe in X and would all be solved by just believing not-X, you might strongly tend to just make that update.

Instead you can do it piecemeal. You mend one rend today, one tomorrow. Any given contradiction can be smoothed over
- by coming up with just slightly implausible explanations,
- by just slightly questioning some logical links,
- by just slightly revising your concepts about X to take a little dent from the contradictory impingement (without actually propagating those modifications to other applications of X, of course),
- by installing slight ugh fields and sleepiness-on-demand and boredom-on-demand and other emotional reactions to steer your attention away,
- by modifying your environment (e.g. not talking to such and such type of person as much, because they might ~~bring up things related to the contradiction~~ be a bit exhausting / needlessly provocative),
- by letting the bailey contract a bit back into the motte just for today,
- by doing a bit more training on pretending to acknowledge the falseness of X just to save face but without internally modifying X-related content,
- etc.
These can be done in small ways, in covert ways, in forgettable ways. They can get compiled into habits without deliberate choice. They are below the threshold where you or anyone else notices that there’s this big pattern.

But they can add up to what is functionally a distorted world, constructed by an auto-sum-threshold attack.

TsviBT 11 Mar 2026 6:23 UTC
2 points
0
in reply to: Lorxus’s comment on: Promoting enmity and bad vibes around AI safety
Well, I meant that to be included under “has really convinced themselves”, where you’re proposing that they could have convinced themselves unconsciously. (Which I agree happens, via a bunch of little ugh fields and piecemeal distorted-world construction.)

Feel free to make an edit to clarify though, it’s a wiki!

TsviBT 11 Mar 2026 6:14 UTC
2 points
0
in reply to: Andrew_Critch’s comment on: Promoting enmity and bad vibes around AI safety
Ok, thanks for the clarification. I think I see a bit more what you’re getting at.

But, rereading Yudkowsky’s tweet, I think your description

performatively “helping” Hegseth by “warning” him that AI leaders would “discard him like used toilet paper”

is not accurate. I think the group being addressed in the tweet is “political leaders of the world”. Admittedly, the tweet is confusingly worded in this regard, and I had to reread it. In the beginning, he says

You have proven that you stand between AI labs and the nice thing they were getting for all their hard work.

and later

Sam Altman does not now look more powerful because you crushed his competitor. He looks less important because you, politicians, crushed his competitor, and did so in a way that made clear that Altman would have to take the orders of any Trump appointee as well.

I think he’s using that as a rhetorical device to express “there’s a conflict here between [AI leaders as a group] and [political leaders as a group]”, because (he claims) that’s how AI leaders are taking it. In this light, I think he’s trying to alert B (political leaders in general) that A (AI leaders) are their enemy.

TsviBT 11 Mar 2026 5:46 UTC
4 points
2
on: Promoting enmity and bad vibes around AI safety
Why is this so downvoted (in addition to upvoted)? It makes clearly stated interesting points on an important topic. Argue or update.

TsviBT 9 Mar 2026 21:47 UTC
7 points
1
in reply to: Lorxus’s comment on: Promoting enmity and bad vibes around AI safety
I wrote the wiki entry on it :) https://www.lesswrong.com/w/sinclair-s-razor

TsviBT 9 Mar 2026 19:58 UTC
24 points
14
in reply to: Thomas Larsen’s comment on: Thomas Larsen’s Shortform
I would guess that debates would be more productive if we more explicitly focused on the ontology/framing that each other are using to reason about the situation, and then discussed to what extent that framing captures the dynamics we think are important.

I strongly agree with this. However, I’ll note as one aspect of the discourse problem, that, at least in my personal experience, people are not very open to this. People’s eyes tend to glaze over. I do not mean this as a dig on them. In fact, I also notice this in myself; and because I think it’s important, I try to incline towards being open to such discussions, but I still do it. (Sometimes endorsedly.)

Some things that are going on, related to this:
- It’s quite a lot of work to reevaluate basic concepts. In one straightforward implementation, you’re pulling out foundations of your building. Even if you can avoid doing that, you’re still doing an activity that’s abnormal compared to what you usually think about. Your reference points for thinking about the domain have probably crystallized around many of your foundational concepts and intuitions.
- Often, people default to “questioning assumptions” when they just don’t know about a domain but want to sound smart / don’t want to try to do the difficult work of understanding the domain. That can be tiring / irrelevant for an expert.
- The Criteria for concepts being good are quite muddled and difficult, at least AFAIK.
(Cf. https://www.lesswrong.com/posts/TNQKFoWhAkLCB4Kt7/a-hermeneutic-net-for-agency )

I think it would be good if more people say things like “I think that’s a bad concept, because it obscures consideration X, which is important for thinking about the situation”.

Totally agree, but I think it’s pretty difficult to explain these things. Part of what’s going on is that, if I have concept X and you don’t, and therefore you don’t think about Y as well as you could, that doesn’t mean I can justify X to you, necessarily. You probably have alternate concepts to partially think about Y. For one thing, maybe your concepts actually are as good or better than my X! In which case I should be trying to learn from you, not teach you. For another thing, your specific pattern of thinking about Y in a partially-correct but impoverished way is particular way of being bad (“each unhappy family...”). So, I would have to track your specific errors / blindspots, in order to make a clear + concise case to you that you should use X. (This is a scenario where live convo is just strictly better than text walls.)

TsviBT 9 Mar 2026 19:35 UTC
4 points
0
in reply to: Thane Ruthenis’s comment on: Promoting enmity and bad vibes around AI safety
Well, the parenthetical says “(though it could in theory come with benefits too)”; you could be making a good case for that happening in practice. My statement is just agreeing with the OP that there is a big cost pathway there—even in your example, there’s a bunch of bad stuff that happens from increased symmetric enmity; e.g. it would probably become harder to help A see the error of their ways, the B team would probably end up doing a bunch of enemy stuff purely for the sake of enmity which doesn’t actually help much, A might become B’s enemy and then successfully smack down B, both sides would be spending resources on conflict rather than doing anything in the intersection of ~all human values (such as for example A spending more resources on thinking and getting a better understanding of how their plans are bad for the world / for B), etc. I think the OP is trying to say this narrow point, e.g.:

[...] I think those activities are increasing AI risk, including but not limited to extinction risk. However, that’s a stronger claim than I intend to argue here. Rather, I’ll just be presenting a simple and harmful causal pathway [...]