You claim that Terry Tao says that Benford’s law lack’s a satisfactory explanation. That is not correct. Terry Tao actually gave an explanation for Benford’s law. And even if he didn’t, if you are moderately familiar with mathematics, an explanation for Benford’s law would be obvious or at least a standard undergraduate exercise.
Let X be a random variable that is uniformly distributed on the interval [m⋅ln(10),n⋅ln(10)] for some m<n. Let Y denote the first digit of eX. Then show that P(Y=k)=log10(k+1k).
For all n, let Xn be a uniformly distributed random variable on the interval [an,bn]. Let Yn denote the first digit of eXn. Suppose that limn→∞bn−an=+∞. Prove that limn→∞P(Yn=k)=log10(k+1k).
Terry Tao in that blog post lists the characteristics that imply Zipf’s law here:
“These laws govern the asymptotic distribution of many statistics Q which
(i) take values as positive numbers;
(ii) range over many different orders of magnitude;
(iiii) arise from a complicated combination of largely independent factors (with different samples of Q arising from different independent factors); and
(iv) have not been artificially rounded, truncated, or otherwise constrained in size.”
Terry Tao merely stated that Benford’s law cannot be proven the same way mathematical theorems are proven; “Being empirically observed phenomena rather than abstract mathematical facts, Benford’s law, Zipf’s law, and the Pareto distribution cannot be “proved” the same way a mathematical theorem can be proved. However, one can still support these laws mathematically in a number of ways, for instance showing how these laws are compatible with each other, and with other plausible hypotheses on the source of the data.”
You claim that Terry Tao says that Benford’s law lack’s a satisfactory explanation. That is not correct. Terry Tao actually gave an explanation for Benford’s law. And even if he didn’t, if you are moderately familiar with mathematics, an explanation for Benford’s law would be obvious or at least a standard undergraduate exercise.
Let X be a random variable that is uniformly distributed on the interval [m⋅ln(10),n⋅ln(10)] for some m<n. Let Y denote the first digit of eX. Then show that P(Y=k)=log10(k+1k).
For all n, let Xn be a uniformly distributed random variable on the interval [an,bn]. Let Yn denote the first digit of eXn. Suppose that limn→∞bn−an=+∞. Prove that limn→∞P(Yn=k)=log10(k+1k).
Terry Tao in that blog post lists the characteristics that imply Zipf’s law here:
“These laws govern the asymptotic distribution of many statistics Q which
(i) take values as positive numbers;
(ii) range over many different orders of magnitude;
(iiii) arise from a complicated combination of largely independent factors (with different samples of Q arising from different independent factors); and
(iv) have not been artificially rounded, truncated, or otherwise constrained in size.”
Terry Tao merely stated that Benford’s law cannot be proven the same way mathematical theorems are proven; “Being empirically observed phenomena rather than abstract mathematical facts, Benford’s law, Zipf’s law, and the Pareto distribution cannot be “proved” the same way a mathematical theorem can be proved. However, one can still support these laws mathematically in a number of ways, for instance showing how these laws are compatible with each other, and with other plausible hypotheses on the source of the data.”