From what I gathered, determinism is the norm, not the exception. (I mean identical results within fixed conditions, not invariance across versions). HF transformers, vLLM, TensorRT-LLM. At least for inference. Deepseek v4′s inference code is fully deterministic, even batch-invariant.
This is quite straightforward to test as well: For a given inference setup, you can run the same prefill and decode runs repeatedly, even across different hardware. I did that for my paper: https://arxiv.org/abs/2606.00279v2
I found that only atomic operations with race conditions result in true non-determinism, and as you can read from Deepseek’s technical report, the incentive seems to be towards ensuring determinism, not breaking it. This is for RL stability, or training stability in general.
Reduction trees are pretty much the only thing you need to know for reproducibility, and that knowledge requires next to no disclosure of IP (when it comes to cuBLAS, which is what I worked with).
Conceded on deterministic inference. You know that layer better than I do, and it’s less of a burden than I thought.
Training is the harder half, and by your own description, a different beast. Determinism isn’t the default. The steps are stateful, so there’s no cheap sampled re-check to fall back on. But suppose that gets solved too. You still haven’t answered Fowler on false positives. The covert-adversary argument (detection × cost > gain, one hit is enough) only works if a flag means cheating. Here, the detector is inspector agents (5.2.3), which throw false positives.
That changes the cheater’s strategy. Rather than evade your samples, they can sit inside the noise band and let the false alarms teach your verifier to wave anomalies off. Which is the §2b attribution problem again, still the least-specified part of the proposal. I don’t think it belongs with the “what to do with a flag” politics you bracketed, either. The false-positive rate is a number the detector itself produces, and it limits what any escalation procedure could attribute, regardless of who runs it. The fallback to post-hoc audits on finished models is where the ambiguity is worst.
Capture looks like an engineering problem, at least. I don’t see what you’d build to fix attribution.
From what I gathered, determinism is the norm, not the exception. (I mean identical results within fixed conditions, not invariance across versions). HF transformers, vLLM, TensorRT-LLM. At least for inference. Deepseek v4′s inference code is fully deterministic, even batch-invariant.
This is quite straightforward to test as well: For a given inference setup, you can run the same prefill and decode runs repeatedly, even across different hardware. I did that for my paper: https://arxiv.org/abs/2606.00279v2
I found that only atomic operations with race conditions result in true non-determinism, and as you can read from Deepseek’s technical report, the incentive seems to be towards ensuring determinism, not breaking it. This is for RL stability, or training stability in general.
Reduction trees are pretty much the only thing you need to know for reproducibility, and that knowledge requires next to no disclosure of IP (when it comes to cuBLAS, which is what I worked with).
Conceded on deterministic inference. You know that layer better than I do, and it’s less of a burden than I thought.
Training is the harder half, and by your own description, a different beast. Determinism isn’t the default. The steps are stateful, so there’s no cheap sampled re-check to fall back on. But suppose that gets solved too. You still haven’t answered Fowler on false positives. The covert-adversary argument (detection × cost > gain, one hit is enough) only works if a flag means cheating. Here, the detector is inspector agents (5.2.3), which throw false positives.
That changes the cheater’s strategy. Rather than evade your samples, they can sit inside the noise band and let the false alarms teach your verifier to wave anomalies off. Which is the §2b attribution problem again, still the least-specified part of the proposal. I don’t think it belongs with the “what to do with a flag” politics you bracketed, either. The false-positive rate is a number the detector itself produces, and it limits what any escalation procedure could attribute, regardless of who runs it. The fallback to post-hoc audits on finished models is where the ambiguity is worst.
Capture looks like an engineering problem, at least. I don’t see what you’d build to fix attribution.