Buck answers Is the output of the softmax in a single transformer attention head usually winner-takes-all?

[Buck]

IIRC, for most attention heads the max attention is way less than 90%, so my answer is “no”. It should be very easy to get someone to make a basic graph of this for you.