Joachim Bartosik

Karma: 206

Joachim Bartosik 10 Mar 2026 22:53 UTC
1 point
0
in reply to: Joachim Bartosik’s comment on: On The Independence Axiom
Ah. I misunderstood. I missed that the uncertainty is on a different thing

Joachim Bartosik 10 Mar 2026 18:13 UTC
1 point
0
on: On The Independence Axiom
A clever adversary who knows your preferences can now exploit this. They offer you a sequence of trades: pay a small amount to switch from plan-A to plan-B before the coin flip (because ex ante you prefer B in context), then after the coin lands heads, pay a small amount to switch from B to A (because ex post you prefer A in isolation). You have paid twice and ended up exactly where you started.
I feel confused by this example.
If I understand correctly, picking a more specific example.

t0: someone decides to toss a biased coin making it more likely it lands tails. If it lands heads you put 1M$ in box H. If it lands tails they put it in box T.
Adversary offers me (i) i’ll take H, free of charge; or (b) I’ll take T but I have to pay you 0.01$.

Then after the coin toss you offer me make a switch for 0.01$.
I this doesn’t sound like a money pump.
If I knew offer 2 is coming then I’m simply silly if I take the cheaper option, I’m plain throwing money away for no reason.
If I didn’t know then it seems I’m paying for options / lack of information not form making bad decissions.

Joachim Bartosik 4 Mar 2026 13:18 UTC
2 points
1
on: A Tale of Three Contracts
I do genuinely believe that Altman was trying to de-escalate the situation, whether or not he had or is having the intended effect.
I don’t understand why you think that.
I think Altman is very competent in this kind of thing and it seems like there were much better moves available if the goal was deescalating (I think you note some in the post).
To me behavior here (quickly signing a contract which supports DoWs story and improves their BATNA) is more consistent with motivations different than wanting to deescalate.

Milder temperature makes a hell stable

Joachim Bartosik3 Mar 2026 22:25 UTC

16 points

1 comment2 min readLW link

Joachim Bartosik 23 Feb 2026 18:02 UTC
1 point
0
in reply to: Steven Byrnes’s comment on: Why we should expect ruthless sociopath ASI
I didn’t try to claim ‘“intrinsically caring about other people” is not a thing that exists in humans’. It does. I also think it’s not strong enough to do that much vs other incentives. It’s strong enough to in some places establish a system which in a buch of situations aligns other incentives with the drive.
And replying to your example: parent-child instincts are real so I’d expect Alice to care. But if roles were reversed a lot of people decide to not take care of their parents

Joachim Bartosik 23 Feb 2026 15:44 UTC
1 point
0
in reply to: Steven Byrnes’s comment on: Why we should expect ruthless sociopath ASI

I can’t say I agree with what you wrote.
I also don’t want to move the subject of the discussion from the one you intend.
If you think I’m doing that I’m happy to delete the rest of this comment.
I think it’s important to the topic you want to discuss that a lot of people at a lot of places for a lot of time did a lot of very cruel things to not just their enemies. Instead they did it because it was convenient or fun (including their friends).
I’ll skip details until requested.
To me it seems it’s opposite—very specific arrangement of incentives manages to limit human to being cruel mostly to enemies and those out of sight.
Which is relevant to figuring out how likely it is that ais will be ruthless sociopaths. I would be more optimistic about that if ~100% minds so far were not one (as opposed to 10-90% (quick estimate on how many people historically seem to have engaged in practices that indicate callous disregard for life of others as a matter of routine or entertainment)

Joachim Bartosik 23 Feb 2026 14:37 UTC
1 point
0
on: Why we should expect ruthless sociopath ASI
And hey, what a coincidence, ≈100% of those minds are not ruthless sociopaths.
I don’t think that’s true in any interesting way. See a lot of examples[1] from history of humans doing terrible things to other humans and also everything else. And how little care about this.
Boring way to defend this is “but if we exclude the stuff people don’t care about”. But then what makes you think your welfare is on the list of the stuff AI cares about?

[1] Cesar in Gaul, Belgian Kongo, WWI, Soviet Union, factory farming

Joachim Bartosik 11 Feb 2026 13:25 UTC
1 point
0
on: Claude Code #4: From The Before Times
Ado offers an introductory guide to bash, for those who don’t know.
find is not exactly safe it has -exec flag which executes commands

Joachim Bartosik 31 Jan 2026 21:53 UTC
1 point
0
on: AI #153: Living Documents
That would include Google CEO Sundar Pichai, since without his support if Demis tried to do this he would presumably be replaced.
Larry Page and Sergey Brin hold majority voting control of Alphabet. Sergey has been actively involved in Gemini development. In 2014 Google created non-voting Class C shares and switched new employee equity grants to that class to prevent dilution of their control (SEC filing). So Pichai may not be the relevant decision-maker.
Page called Musk a ‘speciesist’ for prioritizing human interests over future digital minds (Business Standard).

Joachim Bartosik 23 Jan 2026 11:53 UTC
9 points
3
on: AI #152: Brought To You By The Torment Nexus
pioneer self-driving
Tesla didn’t get into the space early, didn’t deliver yet (unlike others), doesn’t seem to have made significant contribution to the field. So I wouldn’t describe Musk as pioneering self-driving cars.
Waymo predecessor has been working on it in 2009[1], discussed the project publicly in 2010[1]. In 2012 they’ve reported 300k self-driven miles[2]. That’s all before first public discussion of self driving by Musk in 2013[3].
And now Waymo is doing actual self driving while Teslas are doing is not full self driving, but level 2 autonomy, available in many other brands (usually called something adaptive cruise control + line assist)[4], basically not yet where Waymo predecessor has been in 2015[1].

[1] https://en.wikipedia.org/wiki/Waymo
[2] https://canvasbusinessmodel.com/blogs/brief-history/waymo-brief-history
[3] https://en.wikipedia.org/wiki/Tesla_Autopilot
[4] https://en.wikipedia.org/wiki/Advanced_driver-assistance_system

Joachim Bartosik 12 Jan 2026 9:30 UTC
1 point
0
on: AI #150: While Claude Codes
The main reason the ‘AGI types’ are not calling for tax cuts is, quite frankly, that we don’t much care. The world is about to be transformed beyond recognition and we might all die, and you’re talking about tax cuts and short term consumption levels?
I would also say that if you expect transformation by AGI to result in human extinction it seems bad to make worlds where don’t do the transformation worse. (as it makes it less likely we’re going to do the deadly transformation because otherwise we’d have to face unpleasantness). And how much good can you buy with the extra debt?
This seems similar to personal considerations on getting in debt as much as possible adn spending all the money now.

Joachim Bartosik 30 Dec 2025 17:35 UTC
2 points
1
on: Dating Roundup #9: Signals and Selection
The Secretary Problem thus suggests that if you are maximizing, you should be deeply stingy about accepting a match until you’ve done a lot of calibration, and then take your sweet time after that.
Also you have much richer information than in the secretary problem, so you can do way better! https://putanumonit.com/2019/03/03/exponential-secretary/ has a nice write up on this.

Joachim Bartosik 30 Dec 2025 15:35 UTC
1 point
0
on: Dating Roundup #8: Tactics
This is fine if you’re shooting a bunch of shots, but not if this is an especially valuable shot to shoot.
So taking a poorly executed shots is evidence that you’re taking many shots, which affects chances of getting a no (esp. if you don’t know each other well—if you do this is likely not very much evidence).

Joachim Bartosik 16 Dec 2025 13:35 UTC
4 points
0
on: Monthly Roundup #37: December 2026
so they become afraid to speak out or otherwise try to help the village win, and that the villager side is in various ways the harder one to play well.
In my experience playing games like this being too good at being a a villiger has a few disadvantages:
- This is an iterated game, when you play a werewolf in the future it will be very obvious,
- Villigers tend to eliminate you real quick (possibly because you acting this way makes it plain they’re not? I’m not sure)
- If you can play as a werewolf in the future and mask this well it might imply things about you you don’t want to imply (or maybe you do? something something motive ambiguity? I think benjaminrosshoffman.com had a good post that would be relevant but I have only a few minutes so I thought I’d drop a quick note rather than forget about this)

Joachim Bartosik 12 Nov 2025 12:56 UTC
1 point
0
in reply to: Anon User’s comment on: Do not hand off what you cannot pick up
I think I can see how this might scale.

The way this looks to me is that if you’re applying this consistently in an organization, you don’t need to actually fully do all tasks that need doing. You need to be able to recurse 1 level (which if you actually do might involve going down a level… but you mostly don’t need to go down 1 level, going down 2 levels is much more rare, etc.).

To use your example: low-level tasks should not be bubbling up to CEO level. If a controversy about naming a variable bubbles up from a code review to a CEO of a company with 100k people—clearly there has been a failure on multiple levels in the middle (even if the CEO is not up to date on the style guide for the language). The CEO might make the call but more importantly they need to do something about the suborganization before it blows up.

But I’d like to know if this is how Lightcone sees scaling of this principle.

Joachim Bartosik 5 Nov 2025 18:41 UTC
1 point
0
on: Crime and Punishment #1
~~Is this not being posted to substrack intentional?~~
EDIT: It’s been published there in April

Joachim Bartosik 28 Oct 2025 12:29 UTC
1 point
0
on: AI #139: The Overreach Machines
Except at Stanford and some other colleges you can’t, because of this thing called the ‘honor code.’ As in, you’re not allowed to proctor exams, so everyone can still whip out their phones and ask good old ChatGPT or Claude, and Noam Brown says it will take years to change this. Time for oral exams? Or is there not enough time for oral exams?
I’m very confused by this. How are LLMs the problem here? Sounds like you could have been googling answers, calling human helpers on your mobile for last decades?
And bring books / notes with you before that?

Joachim Bartosik 30 Sep 2025 11:10 UTC
2 points
0
on: On Dwarkesh Patel’s Podcast With Richard Sutton
- Almost all feedback is noisy because almost all outcomes are probabilistic.
Yes but signal / noise ratios matter a lot.
Language is somewhat optimized to pick up signal and skip noise. For example “red” makes it easier to pick ripe fruit, “grue” doesn’t really exist because its useless, “expired” is a real concept because it’s useful.
It also has some noise added. For example putting (murderers and jay wakers) in a category “criminal” to politically oppose something.
Also not being exposed to the kind of noise that’s present IRL might be an issue when you start to deal with IRL (sometimes people say something like “just do the max EV action” is a good enough plan)
I’m pretty sure this is some obstacle for LLMs, I’m pretty sure its something that can be overcome, I’m very unsure how much this matters.

Joachim Bartosik 29 Sep 2025 10:27 UTC
2 points
0
on: Economics Roundup #6
The obvious follow-up question is why is there not epic capital flight by every dollar that isn’t under capital controls? Who would ever invest in a Chinese company if they had a choice (other than me, a fool whose portfolio includes IEMG)? Certainly not anyone outside China, and those inside China would only do it if they couldn’t buy outside assets, even treasuries or outside savings accounts. No reason to stick around while they drink your milkshake.
If I understand correctly basically all money is under controls—individuals can buy only 50′000 USD worth of foreign currencies per year.
It used to be possible to exchange more by exchanging yauns for casino chips, playing and exchanging winning chips for USD but ~2021 china cracked down on this (eg. sending someone to prison for 18 years for running an operation like that) and it’s harder now.

Joachim Bartosik 26 Aug 2025 10:33 UTC
4 points
0
on: Quality Precision
This situation puzzles me. On the one hand, I feel a strong logical compulsion to the first (higher total utility) option. The fact that the difference is unresolvable for each person doesn’t seem that worrying at a glance, because obviously on a continuous scale resolvable differences are made out of many unresolvable differences added together.
On the other hand, how can I say someone enjoys one thing more than another if they can’t even tell the difference? If we were looking at the lengths of strings then one could in fact be longer than another, even if our ruler lacked the precision to see it. But utility is different, we don’t care about the abstract “quality” of the experience, only how much it is enjoyed. Enjoyment happens in the mind, and if the mind can’t tell the difference, then there isn’t one.
It seems to me like your own post answers this question?
Any individual is unlikely to notice the difference, but if we treat those like ELO[1] ChatGPT tells me ELO 100 wins 50.14% of the time. Which is not a lot, but with 1 mllion people thats on average some 2800 people more saying they prefer 100 option than 99 option.
[1] Which might not be right, expected utility sounds like we want to add and average utility numbers and it’s not obvious to me to do stuff like averaging ELO.

Joachim Bartosik

Milder tem­per­a­ture makes a hell stable

Milder temperature makes a hell stable