Mis-Understandings

Karma: 227

Mis-Understandings 2 Nov 2025 15:53 UTC
1 point
0
in reply to: quetzal_rainbow’s comment on: Decision theory when you can’t make decisions
One box P1 and Two box P1 are not the only strategies. You can also try to find some source of randomness to take a mixed strategy from. If you are not able to take mixed strategies, Game theory and decision theory are not guaranteed to converge.

Mis-Understandings 2 Nov 2025 1:31 UTC
7 points
2
on: Decision theory when you can’t make decisions
How does the typical construction of Newcomb’s problem handle mixed strategies?
1. The predictor can read your RNG, so your strategy is not really mixed. But this is a violation of game theory assumptions about the possibility of mixed strategies. If you are banned from mixed strategies, Game theory does not work so well, as I understand it.
2. The predictor has some utility distribution over (One box, two box) (Predict one box, Predict two box), and you can solve the Nash equilibrium, maybe to a mixed strategy equilibrium (where you pick one box the percentage) (IE. you want to one box often enough that the Predictor predicts that you will, but no more than that, based on the relative payouts)
3. The predictor has some policy on P(Place in box) for each P (onebox), and there the maximum utility might not be at P(onebox)

Mis-Understandings 14 Oct 2025 0:54 UTC
2 points
−1
on: If Anyone Builds It Everyone Dies, a semi-outsider review
The question of why no “might kill us” as a class is simple. There is such a class, but if it lost the fight to kill us, it obviously was not ASI (picking a fight with the world and losing is pretty dumb), or it might win, at which case it won, we die. And then we will be in the same scenario for every AI stronger than it, and for AI weaker than it that might yet get lucky, just as we might get lucky and win at bad odds. The next AI we make will also want to fight us for the same reasons, and we will need to either fight it to (including preemptively, e.g. turning it off because a dumber model did something), or get a reason to believe that we will never fight it. And if you know you will fight your AI eventually, and you will win now, fight now.

Mis-Understandings 6 Oct 2025 14:04 UTC
11 points
9
on: LLMs are badly misaligned
if it were competent enough to reflect and actively reconcile its own inner drives
Why do we think that reflection is neccesary for competence. That is, competence does not seem to imply coherence, unless I missed an argument.

Mis-Understandings 3 Oct 2025 11:26 UTC
29 points
0
on: No, That’s Not What the Flight Costs
This analysis is slightly incorrect, since if rewards programs are treated as assets, they contribute to net enterprise value, not to market cap. Market cap mostly subtracts loans outstanding from the total expected returns of the enterprise. American, for instance, has 37 billion dollars out in debt, and the market cap is the expected value above the expected payout to the loans.
So in this analysis, American is actually worth 47 billion-ish dollars. 37 billion of those are owned by the creditors, and 10 billion is owned by the shareholders. 22 billion of that worth is the rewards program, and the rest is the rest of the business.
If american’s rewards program disappeared, the creditors would take a haircut, but they would not get nothing.
You can do the same for all of these.
As an aside, every time I have seen someone saying this, they neglect to model debt. I think it might be systematically confusing.
What links here?
- Halfhaven Digest #1 by Taylor G. Lunt (8 Oct 2025 14:24 UTC; 15 points)

Mis-Understandings 18 Sep 2025 15:27 UTC
1 point
0
on: AI #134: If Anyone Reads It
, better known as Anarchy, State and Utopia by Robert Nozick.
Broken link?

Mis-Understandings 12 Sep 2025 17:20 UTC
3 points
2
on: AI #131 Part 2: Various Misaligned Things
Is this a duplicate? Posted 29th Aug 2025 and Today, and is not a mirror of a repost on Don’t Worry About the Vase

Mis-Understandings 26 Aug 2025 20:24 UTC
2 points
0
on: Hidden Reasoning in LLMs: A Taxonomy
For these hidden reasoning steps, especially the epiphenomenal model, there can also be redundancy between weight computation and chain of thoughts. That is, a dumb model seeing the chain of thought of a smart model might get the right answer when it would not otherwise, even if the chain of thought does not help the large model.
Under the assumption of separate initialization, this probably does not happen in cases where the extra information is stenographic, or in some examples of irrelevant reasoning because things are not being passed through the token bottleneck.

Mis-Understandings 12 Aug 2025 13:45 UTC
6 points
1
on: GPT-5s Are Alive: Outside Reactions, the Router and the Resurrection of GPT-4o
Steve Strickland: GPT-5 is the first model I’ve used that will deliberately give a wrong answer to ‘check you’re paying attention’.
This fundamentally unreliable technology is not going to put us all out of work.
Is that intentional, or is the name a hallucination and the cheeky comment an intervention aimed on exploiting the fact that models can tell when they are hallucinating.

Mis-Understandings 30 Jul 2025 20:08 UTC
1 point
0
on: Childhood and Education: College Admissions
In lots of these cases, it seems like every university ends up thinking (This student would be making a mistake to commit to our university). It seems like a really hard sell to say universities should accept people who they think would be making a mistake to actually go. This is unfair, the university does not know your other options, and this mistake framework requires that committing be worse than the decision that you would actually take.

Mis-Understandings 20 Jun 2025 22:02 UTC
3 points
0
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform
I think this might be an attempted countermeasure against prompt injection. That is, it wants to mix autoregressive and reconstructed residuals. Otherwise, it might lose it’s train of thought (end up continuing the article not following the prompt).

Mis-Understandings 16 Jun 2025 23:13 UTC
2 points
0
on: RTFB: The RAISE Act
There is a hidden legal standard that this law would like to endorse about existing laws, but I am not sure that it sets out. It is at least suggesting a precedent when it talks about “Things that would be crimes requiring intent”, in that there is an argument that LLMs or any AIs do not yet have the requisite mental state, since they don’t really have mental states. So they can’t be liable for crimes becasue of mens rea reasons, and since you did not know you can’t have intent. This law is trying to argue that that is basically bullshit.
(i) Acts with no meaningful human intervention; and
(ii) Would, if committed by a human, constitute a crime specified in the penal law that requires intent, recklessness, or gross negligence, or the solicitation or aiding and abetting of such a crime.
seems to imply that the lawmakers believe that there should not be a way for using an AI to sever liability, criminal or civil, for an action, regardless of what you intended the AI do, and if you are in a position to do that this makes it the providing companies problem. They fucked up, but it informs the prosecutor that they should prosecute, because the provider fucked up.
Basically, it is 1 law away from doing the “if your dog commited a violation, x happens”, for “If an AI commits a tort, who is liabile” with an answer that is not nobody. There is an argument under current law that there is a level of independence where that is nobody, because the AI can’t have relative intent. This law tries to say “IT REALLY SHOULD BE THE AI COMPANY” if liability would sink in an independent agent without it’s own assets.

Mis-Understandings 15 Jun 2025 18:57 UTC
17 points
14
in reply to: Cole Wyeth’s comment on: Intelligence Is Not Magic, But Your Threshold For “Magic” Is Pretty Low
We are exactly worried about that though. It is not that AGI will be inteligent (that is the name), but that it can and probably will develop dangerous capabilities. Inteligence is the word we use to describe it, since it is associated with the ability to gain capability, but even if the AGI is sometimes kind of brute force or dumb does not mean that it cannot also have dangerous enough capabilities to beat us out.

Mis-Understandings 12 Jun 2025 18:31 UTC
2 points
0
on: AI #120: While o3 Turned Pro
On AI diplomacy, 1. It is a useful model benchmark but 2. Is not new, since we already had human level full press diplo harnesses (CICERO) (2022). It would be useful to compare these new harnesses to the more targeted system.

Mis-Understandings 22 May 2025 14:17 UTC
2 points
0
on: AI #117: OpenAI Buys Device Maker IO
whether
Is that supposed to be weather?

Mis-Understandings 17 May 2025 13:59 UTC
1 point
0
in reply to: leogao’s comment on: Mis-Understandings’s Shortform
No, because you need different strategies to prove the loop case (which you can do with just a sequence of transitions), the halt case (the same) and the use the infinite amount of memory case.
There is no proof that will catch 100% of case 3, (because then you would have a halting oracle). But you can create a program that will halt iff a program halts, or halt iff it loops in finite states or halts. (I could write it, but it just runs real slow). You cannot write the program that halts if another program uses an infinite amount of memory (since then you could build a halting oracle). There are NO halting oracles, not just no efficient halting oracles.

Mis-Understandings 16 May 2025 19:17 UTC
1 point
0
on: Slow corporations as an intuition pump for AI R&D automation
Random Thought. Automated Corp has a real advantage. That is, inference and training can run on the same GPUs, (to a first approximation). So for slow corp, if they spend a day deciding, and don’t commit to runs, they are wasting GPU time. But the other corps don’t have this problem. It is a big problem. There is something there, the real question is (How much does thinking about the results of your test queue improve the informational value of the tests you run).

Mis-Understandings 16 May 2025 2:58 UTC
5 points
0
on: Mis-Understandings’s Shortform
A program must either loop, halt, or use an infinite amount of memory.
Halting is almost never the highest EV action for any goal
Looping mught be high EV in some cases and some goals, but I would not expect many of them, and definitely not short loops (the longer the loop, the more like case 3).
So the policy (Always take the highest EV action for a particular goal (that is the danger model)) is a program that will use an infinite amount of memory, since it neither halts nor loops.

Mis-Understandings 11 May 2025 17:44 UTC
1 point
0
on: Better Air Purifiers
Thoughts about HEPA is as a standard. A lower percentage removed can happen in two ways. One is probabilistic, where the same particle might or might not get trapped. Managing probabilistic capture, total circulation and fan stats (airflow vs static pressure) is probably a good idea. Introducing determinsitic non-captures (where there are a a class of particles not captured), can be a problem, as those will not be affected by the purifier. But that is engineering that requires only dilligence.
Another way of putting it is that it makes more sense to use higher airflow, lower static pressure fans, and the filters should work with that.

I agree that the correct measure is particulates where people breathe, not simply exhaust particulates.

Mis-Understandings 8 May 2025 14:59 UTC
3 points
0
in reply to: plex’s comment on: ete’s Shortform
A contextualization of people toting big personal speedup numbers.
People get way more productive by rethinking their workflow, especially in research, not all the time, but like it was not an unprecedented story in 2015.
Do you remember when people were talking about 10x engineers in the 2010s.
Discovering that in a new workflow, you are the 10x engineer is not unprecedented.
The question is the rate of (try new thing)-> clicks with workflow so output jumps, higher.
Sometimes, people got 10x more productive from some change before any of this, so understand that any change in workflow has a noise floor even at these productivity leaps.