I think I have a Lean proof that C > 1.93777676
Archimedes
Karma: 1,127
I don’t think 1.838 is the correct lower bound. Would a Lean proof that it’s greater than 1.9 be meaningful to you?
Separately, how small do you need C to be to be meaningful? Like, is 500 materially different from 1771 or do you need single digits for your purposes?
Interesting that they don’t feel the progress. Maybe schoolwork shaped things were mostly saturated years ago, at least within their abilities to discriminate?
Got it. You were after an explanation more than a defense and you got some good ones.
It’s a broadly adopted term of art that has more specific nuance than “previously unknown” and abbreviates nicely to “0-day”. What more do you want from jargon?
It’s fine to use longer or squishier descriptions if you need to write to the broad audiences, but it’s simply part of the language of the field, just like you used “jargon” instead of something more generic like “technical terminology”. Consider why you chose that word and see if the same arguments apply to “zero day”.
I’d guess that “pedophilia” is largely conflated with CSA, and most people don’t care to disentangle them. So your argument isn’t against a coherent, considered opinion so much as a gut reaction.
I too spent hours the first few times and seconds now.
Even if they don’t learn anything, they’re still getting more opportunities for something to work out well.
So, the customers are the “civilians” in this scenario?
Cross-posting the comic version here too. It’s the only one I’ve actually read in full so far.
Someone made a comic version of this that may be more digestible for some audiences.
The amount by which they differ is quantified in the paper.
From §3.1 “The J-space supports verbal report” (the concept-vector split)
Notably, across concepts and workspace layers, the J-space component carries a median of only 6–7% of the concept vector’s variance, with the remaining ~93% lying outside the J-space.
In the early workspace layers, the top-κ SAE stratum’s gain exceeds that of the J-lens vectors. We take this as further evidence that the J-lens only partially captures the model’s ‘true’ workspace representations: the highest-κ SAE features may approximate the underlying workspace directions more closely than J-lens vectors do, since the latter are constrained to single tokens.
Conceded on deterministic inference. You know that layer better than I do, and it’s less of a burden than I thought.
Training is the harder half, and by your own description, a different beast. Determinism isn’t the default. The steps are stateful, so there’s no cheap sampled re-check to fall back on. But suppose that gets solved too. You still haven’t answered Fowler on false positives. The covert-adversary argument (detection × cost > gain, one hit is enough) only works if a flag means cheating. Here, the detector is inspector agents (5.2.3), which throw false positives.
That changes the cheater’s strategy. Rather than evade your samples, they can sit inside the noise band and let the false alarms teach your verifier to wave anomalies off. Which is the §2b attribution problem again, still the least-specified part of the proposal. I don’t think it belongs with the “what to do with a flag” politics you bracketed, either. The false-positive rate is a number the detector itself produces, and it limits what any escalation procedure could attribute, regardless of who runs it. The fallback to post-hoc audits on finished models is where the ambiguity is worst.
Capture looks like an engineering problem, at least. I don’t see what you’d build to fix attribution.
Re-execution is a much bigger ask than the draft implies. Bit-exact replay is a property the whole stack must hold all at once: the framework, math libraries, collective comms, the compiler’s autotuning, every custom kernel, the entire data pipeline. One nondeterministic component anywhere breaks it. Some ops have no deterministic implementation; much of the stack is closed-vendor code you can’t determinize yourself; and any version bump can silently break bit-exactness. So you either banish the nondeterminism and freeze the whole stack or commit to revalidating it forever. Determinizing one model’s inference is a demonstrated proof of concept. Requiring it across every training run, kernel, and pipeline on a fleet, permanently, is a stack-wide re-engineering nightmare, not just an enormous throughput hit. I’d expect that throttles the majority of real workloads, benign or otherwise.
That tax falls on benign and forbidden compute alike, so it doesn’t single out cheating. It just makes the monitored regime prohibitively slow and expensive for everyone. The consequence is a strong incentive to keep serious work off the monitored hardware. As others have noted, the scheme only governs compute that’s enrolled in the first place. If running under the monitor means giving up the tools and techniques high-performance computing is built on (or fully re-engineering it end-to-end), both benign and forbidden compute migrate to whatever capacity isn’t so severely constrained.
I can’t think of a single term for this, but it can be articulated as is-ought laundering through accountability sinks. The interlocutor describes what the structure is that diffuses local fault and treats that description as if it settles whether the outcome ought to happen.
This is a control theory problem obscured by terminology like “oneshotness”.
I interpret the phenomenon EY is gesturing at as a stability margin failure. That is, a system going off course at a rate that exceeds a controller’s ability to correct. Most of the disagreement is not about this model at a high level, but about how the interaction dynamics play out and what levels of uncertainty to apply.
Controlling the Viking failed immediately upon losing the only correction channel. The control rate going to zero means game over.
The Mars Observer failed slowly as vapor accumulated over 11 months with no sensor detecting it as a problem. Zero control rate for a different reason. This time, the drift off course wasn’t even observed until too late.
The Maginot Line failed because France was miscalibrated on both rates. They assumed the Germans would advance (“off course”) more slowly and that their mobilization (“correction”) would be faster.
ASI fits the pattern but has increased levels of cursedness affecting both rates. An AI can act faster than humans can observe and respond, interfere with corrective mechanisms, and obfuscate observability (e.g., sandbagging and playing the training game). Trying to control a strategic adversarial opponent goes beyond classical control theory with its known engineering techniques into the territory of dynamic games.
The disagreement is not whether there is a level of criticality where the situation is unrecoverable (most reasonable people agree with that), but how fast the AI might take a “sharp left turn” or undergo an RSI loop phase change, as well as how fast humans can adapt scalable oversight and meaningful alignment strategies.
This is not a novel framing. Elija Perrier lays out a more formal description here: Out of Control—Why Alignment Needs Formal Control Theory (and an Alignment Control Stack), and Daniel Kokotajlo is making similar decompositions in other comments. Beren Millidge has a more optimistic take here: Maintaining Alignment during RSI as a Feedback Control Problem.
Let’s drop “oneshotness” and discuss in terms that can be modeled more precisely than debating what counts as “one shot”.
I don’t think you can get away from the baggage. Voters would associate him with Zuckerberg and Meta/Facebook, whether fair or not. The EA branding is also a liability in the political arena post-FTX, whether fair or not.
Does the guy have ANY retail political skill on the record? To me, it seems like a category error considering Moskovitz for POTUS. Someone who deeply understands a problem but has no political experience is better suited as an advisor. What about all the other problems a POTUS has to deal with?
I didn’t realize this was controversial. This is only n=1 evidence (and not necessarily cancerous), but a cleanly controlled natural experiment that visually demonstrates non-sunburn skin damage via consistent asymmetric sun exposure on one side of a truck driver’s face: https://www.nejm.org/doi/full/10.1056/NEJMicm1104059
For something more scientific, maybe these work better?
This is above my head, but my Claude has a comment that might be useful to understanding why your conjecture is false.
Claude Opus 5
The exact-cap version is false. Here’s a witness.
Take the doubly symmetric binary source: X ~ Ber(1/2), Y = X ⊕ N with
N ~ Ber(q), 0 < q < 1⁄2. For the stochastic latent, flip an independent
coin and output X with probability λ, otherwise output ⊥. Then
I(X;Y|V) = (1−λ)·I(X;Y)
I(V;X|Y) = λ·h(q)
I(V;Y|X) = 0
Now look for a deterministic Γ with I(X;Y|Γ) ≤ I(X;Y|V). The constant Γ
gives exactly I(X;Y), so it’s excluded for every λ > 0. And every
nonconstant Γ costs at least h(q): join two atoms of the 2×2 support
whenever they share a row or a column, and that graph is a 4-cycle. A
cycle has no bridge, so any nonconstant partition cuts at least two
edges, each of weight h(q)/2. Γ = X attains it.
So the cheapest admissible Γ costs h(q) while the stochastic redundancy
is λ·h(q). The ratio is 1/λ, unbounded, and independent of q.
The failure is knife-edge, which I think is the useful part. Relax the
constraint to I(X;Y|Γ) ≤ I(X;Y|V) + η for any fixed η > 0 and the
constant Γ becomes admissible as soon as λ ≤ η/I(X;Y), so the supremum
is I(X;Y)/η and finite. The multiplicative form I(X;Y|Γ) ≤ C’·I(X;Y|V)
is untouched, and this family can’t refute it: its ratio is bounded by
C’/(C’−1).
The reason is convexity. A stochastic latent can time-share between
deterministic maps using an independent coin, so its achievable set of
(mixing, redundancy) pairs is convex. A partition can’t time-share. At
exactly equal mixing you’re comparing a convex region against a finite
set of points. So I’d read the sum form as doing real work rather than
being an artifact of the proof.
The natural strengthening that survives is the weighted family: for
λ > 0, compare min over Γ of [λ·I(X;Y|Γ) + H(Γ|X) + H(Γ|Y)] against
inf over V of [λ·I(X;Y|V) + I(V;X|Y) + I(V;Y|X)]. What’s proved now is
the λ = 1 slice, so that’s a genuine strengthening and it dodges the
obstruction above.
Two notes on #2, since GitHub didn’t show check runs on the head. I
replayed 34e3f898 in a clean checkout: source audit, all 8734 build
jobs, assert_no_sorry, and #print axioms pinned to exactly [propext,
Classical.choice, Quot.sound]. It verifies. Worth flagging that
T_le_698 is a T-endpoint, and generalized_stoch_to_det and LW_sum are
still exported at 1771, so the headline number and the public API
aren’t the same statement yet.
On extending past two observables, one caution: the analogous exact-cap
version fails the same way, and the machinery in #2 is specifically
two-factor (HGR maximal correlation, a bilinear form, two-factor
Hölder, a product marginal). It may extend, but I wouldn’t assume it’s
free.