# [Question] Is the sum individual informativeness of two independent variables no more than their joint informativeness?

Is it true that:

If I(X;Y) = 0 then I(S;X) + I(S;Y) ⇐ I(S;X,Y)

Can you find a coun­terex­am­ple, or prove this and teach me your proof?

Some­one showed me a sim­ple an­a­lytic proof. I am still in­ter­ested in see­ing differ­ent ways peo­ple might prove this though.

• For a vi­su­al­iza­tion, see in­for­ma­tion di­a­grams, and note that the cen­tral cell I(S; X; Y) must be non-pos­i­tive (be­cause I(S; X; Y) + I(X; Y | S) = I(X; Y) = 0).

• We want to prove:

This can be rewrit­ten as:

After mov­ing ev­ery­thing to the right hand side and sim­plify­ing, we get:

Now if we just prove that is a prob­a­bil­ity dis­tri­bu­tion, then the left hand side is , and Kul­lback-Leibler di­ver­gence is always non­nega­tive.

Ok, q is ob­vi­ously non­nega­tive, and its in­te­gral equals 1:

Q.e.d.

• Just for amuse­ment, I think this the­o­rem can fail when s, x, y rep­re­sent sub­sys­tems of an en­tan­gled quan­tum state. (The most nat­u­ral gen­er­al­iza­tion of mu­tual in­for­ma­tion to this do­main is some­times nega­tive.)