Another way to think about higher-rank density matrices is as probability distributions over pure states; I think this is what Charlie Steiner’s comment is alluding to.
So, the rank-2 matrix from my previous comment, ρ=12I2 can be thought of as12|0⟩⟨0|+12|1⟩⟨1|
, i.e., an equal probability of observing each of |0⟩,|1⟩. And, because I2=|x⟩⟨x|+|y⟩⟨y| for any orthonormal vectors |x⟩,|y⟩, again there’s nothing special about using the standard basis here (this is mathematically equivalent to the argument I made in the above comment about why you can use any basis for your measurement).
I always hated this point of view; it felt really hacky, and I always found it ugly and unmotivated to go from states |Ψ⟩ to projections |Ψ⟩⟨Ψ| just for the sake of taking probability distributions.
The thing above about entanglement and decoherence, IMO, is a more elegant and natural way to see why you’d come up with this formalism. To be explicit, suppose you have the state |0⟩, and there is an environment state that you don’t have access to, say it also begins in state |0⟩, and initially everything is unentangled, so we begin in the state |00⟩. Then some unitary evolution happens that entangles us, say it takes |00⟩ to the Bell state |00⟩+|11⟩√2.
As we’ve seen, you should think of your state as being 12I2, and now it’s clear why this is the right framework for probabilistic mixtures of quantum states: it’s entirely natural to think of your part of the now-entangled system to be “an equal chance of |0⟩ and |1⟩”, and this indeed gives us the right density matrix. It also immediately implies that you are forced to also allow that it could be represented as “an equal chance of |+⟩ and |−⟩” where |+⟩,|−⟩=|0⟩±|1⟩√2, and etc.
But it makes it clear why we have this non-uniqueness of representation, or where the missing information went: we don’t just “have a probabilistic mixture of quantum states”, we have a small part of a big quantum system that we can’t see all of, so the best we can do is represent it (non-uniquely) as a probabilistic mixture of quantum states.
Now, you aren’t obliged to take this view, that the only reason we have any uncertainty about our quantum state is because of this sort of decoherence process, but it’s definitely a powerful idea.
Yeah, this also bothered me. The notion of “probability distribution over quantum states” is not a good notion: the matrix I is both (|0\rangle \langle 0|+|1\rangle \langle 1|) and (|a\rangle \langle a|+|b\rangle \langle b|) for any other orthogonal basis. The fact that these should be treated equivalently seems totally arbitrary. The point is that density matrix mechanics is the notion of probability for quantum states, and can be formalized as such (dynamics of informational lower bounds given observations). I was sort of getting at this with the long “explaining probability to an alien” footnote, but I don’t think it landed (and I also don’t have the right background to make it precise)
Actually, I have a little more to say:
Another way to think about higher-rank density matrices is as probability distributions over pure states; I think this is what Charlie Steiner’s comment is alluding to.
So, the rank-2 matrix from my previous comment, ρ=12I2 can be thought of as12|0⟩⟨0|+12|1⟩⟨1|
, i.e., an equal probability of observing each of |0⟩,|1⟩. And, because I2=|x⟩⟨x|+|y⟩⟨y| for any orthonormal vectors |x⟩,|y⟩, again there’s nothing special about using the standard basis here (this is mathematically equivalent to the argument I made in the above comment about why you can use any basis for your measurement).
I always hated this point of view; it felt really hacky, and I always found it ugly and unmotivated to go from states |Ψ⟩ to projections |Ψ⟩⟨Ψ| just for the sake of taking probability distributions.
The thing above about entanglement and decoherence, IMO, is a more elegant and natural way to see why you’d come up with this formalism. To be explicit, suppose you have the state |0⟩, and there is an environment state that you don’t have access to, say it also begins in state |0⟩, and initially everything is unentangled, so we begin in the state |00⟩. Then some unitary evolution happens that entangles us, say it takes |00⟩ to the Bell state |00⟩+|11⟩√2.
As we’ve seen, you should think of your state as being 12I2, and now it’s clear why this is the right framework for probabilistic mixtures of quantum states: it’s entirely natural to think of your part of the now-entangled system to be “an equal chance of |0⟩ and |1⟩”, and this indeed gives us the right density matrix. It also immediately implies that you are forced to also allow that it could be represented as “an equal chance of |+⟩ and |−⟩” where |+⟩,|−⟩=|0⟩±|1⟩√2, and etc.
But it makes it clear why we have this non-uniqueness of representation, or where the missing information went: we don’t just “have a probabilistic mixture of quantum states”, we have a small part of a big quantum system that we can’t see all of, so the best we can do is represent it (non-uniquely) as a probabilistic mixture of quantum states.
Now, you aren’t obliged to take this view, that the only reason we have any uncertainty about our quantum state is because of this sort of decoherence process, but it’s definitely a powerful idea.
Yeah, this also bothered me. The notion of “probability distribution over quantum states” is not a good notion: the matrix I is both (|0\rangle \langle 0|+|1\rangle \langle 1|) and (|a\rangle \langle a|+|b\rangle \langle b|) for any other orthogonal basis. The fact that these should be treated equivalently seems totally arbitrary. The point is that density matrix mechanics is the notion of probability for quantum states, and can be formalized as such (dynamics of informational lower bounds given observations). I was sort of getting at this with the long “explaining probability to an alien” footnote, but I don’t think it landed (and I also don’t have the right background to make it precise)