An Attempted Derivation of the Lindy Effect Wikipedia:
The Lindy effect (also known as Lindy’s Law[1]) is a theorized phenomenon by which the future life expectancy of some non-perishable things, like a technology or an idea, is proportional to their current age.
Laplace Rule of Succesion
What is the probability that the Sun will rise tomorrow, given that is has risen every day for 5000 years?
Let p denote the probability that the Sun will rise tomorrow. A priori we have no information on the value of p so Laplace posits that by the principle of insufficient reason one should assume a uniform prior probability dp=Uniform((0,1))[1]
Assume now that we have observed n days, on each of which the Sun has risen.
Each event is a Bernoulli random variable Xi which can each be 1 (the Sun rises) or 0 (the Sun does not rise). Assume that the probability is conditionally independent of p.
The likelihood of n out of n succeses according to the hypothesis p is L(X1=1,...,Xn=1|p)=pn. Now use Bayes rule
I haven’t checked the derivation in detail, but the final result is correct. If you have a random family of geometric distributions, and the density around zero of the decay rates doesn’t go to zero, then the expected lifetime is infinite. All of the quantiles (e.g. median or 99%-ile) are still finite though, and do depend upon n in a reasonable way.
[Thanks to Vlad Firoiu for helping me]
An Attempted Derivation of the Lindy Effect
Wikipedia:
Laplace Rule of Succesion
What is the probability that the Sun will rise tomorrow, given that is has risen every day for 5000 years?
Let p denote the probability that the Sun will rise tomorrow. A priori we have no information on the value of p so Laplace posits that by the principle of insufficient reason one should assume a uniform prior probability dp=Uniform((0,1))[1]
Assume now that we have observed n days, on each of which the Sun has risen.
Each event is a Bernoulli random variable Xi which can each be 1 (the Sun rises) or 0 (the Sun does not rise). Assume that the probability is conditionally independent of p.
The likelihood of n out of n succeses according to the hypothesis p is L(X1=1,...,Xn=1|p)=pn. Now use Bayes rule
P(p|X1=1,...,Xn=1)=P(X1=1,...,Xn=1|p)dp∫P(X1=1,...,Xn=1|p)dp=pndp∫10pndp=pndp1n+1=(n+1)pndp
to calculate the posterior.
Then the probability of succes for P(Xn+1=1|X1=1,...,Xn=1)=∫P(Xn+1|p)P(p|X1=1,...,Xn=1)
=∫10p⋅(n+1)pndp=n+1n+2
This is Laplace’s rule of succcesion.
We now adapt the above method to derive Lindy’s Law.
The probability of rising n+s days and not rising on the n+s+1 day given that the Sun rose n days is
P(Xn:n+s=1,Xn+s+1=0|X1:n=1)=∫10ps(1−p)(n+1)pndp=(n+1)(1n+s+1−1n+s+2)=n+1(n+s+1)(n+s+2)
The expectation of lifetime is then the average
E(Sun rises s more days)=∑∞s=1sn+1(n+s+1)(n+s+2)
which almost converges :o.…
[What’s the mistake here?]
For simplicity I will exclude the cases that p=0,1, see the wikipedia page for the case where they are not excluded.
I haven’t checked the derivation in detail, but the final result is correct. If you have a random family of geometric distributions, and the density around zero of the decay rates doesn’t go to zero, then the expected lifetime is infinite. All of the quantiles (e.g. median or 99%-ile) are still finite though, and do depend upon n in a reasonable way.