While non-deterministic batch calculations in LLMs imply possibility of side channel attacks, so best to run private queries in private batches however implausible an actual exploit might be… if there is any BENEFIT from cross-query contamination, GSD would ruthlessly latch on any loss reduction—maybe “this document is about X, other queries in the same batch might be about X too, let’s tickle the weights in a way that the non-deterministic matrix multiplication is ever so slightly biased towards X in random other queries in the same batch” is a real-signal gradient 🤔
While non-deterministic batch calculations in LLMs imply possibility of side channel attacks, so best to run private queries in private batches however implausible an actual exploit might be… if there is any BENEFIT from cross-query contamination, GSD would ruthlessly latch on any loss reduction—maybe “this document is about X, other queries in the same batch might be about X too, let’s tickle the weights in a way that the non-deterministic matrix multiplication is ever so slightly biased towards X in random other queries in the same batch” is a real-signal gradient 🤔
How to test that?