Naci Cankaya

Karma: 201

Naci Cankaya 12 Aug 2026 16:00 UTC
2 points
0
on: Don’t Build Mindreading
The problem with mindreading is that it is an asymmetric technology that disproportionately helps those who rule without consent. The productivity enhancements are equally beneficial to everyone who can get them, but detecting defection is of marginal benefit to a project you are joyously working on with friends. It will help large corporations predict who is contemplating leaving. It is of great help to a criminal scared that his subordinates will turn state’s evidence. But it is dictators who will benefit the most.
I would like to understand what assumption you are making about mindreading here: Something that can be used on people without their awareness, or something that will require them to plug into something that is obviously a mind-reading machine.

This changes a lot about the power dynamics.

Naci Cankaya 25 Jun 2026 11:13 UTC
1 point
0
in reply to: William Fowler’s comment on: A system overview for near-term, low-trust AI compute verification
“I’m curious if this proposal necessitates forbidding multi-data center training.”

That’s a tricky edge case for the “all information in and out of a comput island is explainable”, for two reasons: One is the extreme information bandwidth, the other is that -as you explained- the links between datacenters would be hard to secure.

Since this is an edge-case and the size of upcoming megaclusters will probably eliminate the need for inter-datacenter pretraining, I expect this to not be an immediate dealbreaker.

If one really did insist on multi-datacenter training, you could still do the following: Each cluster is treated as an “island” whose in-out traffic must be fully explainable. Rather than capturing the whole traffic (which would be expensive), you can retrofit a setup that randomly samples from passively tapped optical fiber links. Basically a “panopticon” in which the prover can not predict which link will be sampled by the verifier, or when.

“(additionally, if multi-data center training isn’t allowable it also seems like you wouldn’t be able to transfer the completed model to a different data center, like one more suited for inference, after training)”

I am not quite following. Why would that be the case? Because the model has not been trained in a verified manner? Inference verification and training verification are different beasts. It probably makes sense to start with the former, and rely on post-hoc audits of models before training verification is ready.

Here is recent work on both, if you are interested: https://amododesign.com/notes/2026-06-23-verification-algorithms/

Naci Cankaya 24 Jun 2026 13:00 UTC
4 points
0
in reply to: Archimedes’s comment on: A system overview for near-term, low-trust AI compute verification
From what I gathered, determinism is the norm, not the exception. (I mean identical results within fixed conditions, not invariance across versions). HF transformers, vLLM, TensorRT-LLM. At least for inference. Deepseek v4′s inference code is fully deterministic, even batch-invariant.
This is quite straightforward to test as well: For a given inference setup, you can run the same prefill and decode runs repeatedly, even across different hardware. I did that for my paper: https://arxiv.org/abs/2606.00279v2
I found that only atomic operations with race conditions result in true non-determinism, and as you can read from Deepseek’s technical report, the incentive seems to be towards ensuring determinism, not breaking it. This is for RL stability, or training stability in general.
Reduction trees are pretty much the only thing you need to know for reproducibility, and that knowledge requires next to no disclosure of IP (when it comes to cuBLAS, which is what I worked with).

Naci Cankaya 23 Jun 2026 14:56 UTC
2 points
0
in reply to: Ninety-Three’s comment on: A system overview for near-term, low-trust AI compute verification
Yes. This is treated as out of scope here, even though it is a very important, upstream problem. The solutions to this are more politics-shaped than they are technology-shaped (as far as I can tell).

Naci Cankaya 5 Mar 2026 14:20 UTC
1 point
0
in reply to: jbash’s comment on: Make Powerful Machines Verifiable
Construct the circuits? Can you explain what you mean by that?

Naci Cankaya 26 Feb 2026 12:58 UTC
3 points
0
in reply to: Gordon Seidoh Worley’s comment on: All hands on deck to build the datacenter lie detector
Yeah, I didn’t say this, but it would be great if there was a dedicated startup! The tech would necessarily need to be open source, but support could be the business model. That, or the startup could simply be non-profit and funded by Longview or similar:
https://www.longview.org/request-for-proposals-on-hardware-enabled-mechanisms-hems-for-ai-verification/
Things are moving fast though, and by next week I’ll know more about some specific company’s ambitions and roadmaps. Can share this when this info is public.

Regardless, more commercial engagement is needed.

Naci Cankaya 20 Oct 2024 17:50 UTC
9 points
0
in reply to: Nathan Helm-Burger’s comment on: [Linkpost] Hawkish nationalism vs international AI power and benefit sharing
Hello Nathan, thank you for your comment! Really looking forward to your post and how it expands on the idea.

Regarding your estimates: I personally do not think that assigning probabilities to preferable outcomes is very useful. On the contrary, one can argue that the worldviews held by influential people can become self fulfilling prophecies. That is especially applicable to prisoner’s dilemmas. One can either believe the dilemma is inevitable and therefore choose to defect, or instead see the situation itself as the problem, not the other prisoner. That was the point we were trying to make.

Rather than suggesting what to do under the assumption that things turn out well, we were arguing for a shift in perspective and what measures could help with increasing the chances of a good outcome of the advent of TAI. Fixing the game, instead of accepting the rules and hoping for the best.

In the disclaimer, when we said that the relevance of the article increases with longer timelines, we did not mean to narrow its applicability down towards a scenario of an agreed upon AI pause. The purpose was mainly to not appear too confident in our assumptions before being proven wrong by unpredictable impacts of the technology.
What links here?
- A path to human autonomy by Nathan Helm-Burger (29 Oct 2024 3:02 UTC; 53 points)