This was what I expected to see, and I believe it’s equivalent to
H(X,Y,Z) = H(X) + H(Y) + H(Z) - I(X;Z) - I(Z;Y) - I(X;Y | Z)
It appears that Z is very artificially constructed—Z is exactly I(X,Y) in the example. Therefore, H(X,Y) = H(X,Y,Z). Since the term I(X,Y | Z) is mutual information about X and Y given Z, that’s just 0. There’s no new mutual information about X and Y that isn’t already in Z. So I believe that we could replace it with +I(X,Y) - I(X,Y,Z), and get inclusion-exclusion.
I’m confused :/
P(X,Y,Z) = P(X,(Y,Z)) = P(X) + P(Y,Z) - P(X;(Y,Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - P(X;(Y,Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - P((X;Y),(X;Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - (P(X;Y) + P(X;Z) - P(X;Y;Z)) = P(X) + P(Y) + P(Z) - P(X;Y) - P(Y;Z) - P(X;Z) + P(X;Y;Z)
By the inclusion-exclusion principle, no?
This was what I expected to see, and I believe it’s equivalent to H(X,Y,Z) = H(X) + H(Y) + H(Z) - I(X;Z) - I(Z;Y) - I(X;Y | Z)
It appears that Z is very artificially constructed—Z is exactly I(X,Y) in the example. Therefore, H(X,Y) = H(X,Y,Z). Since the term I(X,Y | Z) is mutual information about X and Y given Z, that’s just 0. There’s no new mutual information about X and Y that isn’t already in Z. So I believe that we could replace it with +I(X,Y) - I(X,Y,Z), and get inclusion-exclusion.