Vivek Hebbar comments on Toward A Mathematical Framework for Computation in Superposition

[Vivek Hebbar]

In general, computing a boolean expression with $k$ terms without the signal being drowned out by the noise will require $ϵ<1/k$ if the noise is correlated, and $ϵ<1/k^2$ if the noise is uncorrelated.

Shouldn’t the second one be $\frac{1}{\sqrt{k}}$?

the past token can use information that is not accessible to the token generating the key (as it is in its “future” – this is captured e.g. by the attention mask)

Is this meant to say “last token” instead of “past token”?