I think this reveals that the maxent problem you post at the start (maximum entropy constrained by cross-entropy not exceeding entropy) is actually fully specified by the constraint, rather than being sensitive to the expression being maximised. So the maxent is irrelevant. Don’t know what downstream implications that has.
For sure you can’t have cross-entropy constrained by both of HP and HQ if they [ETA that is, P and Q, not their entropies] differ at all. Maybe you’d make some headway if you introduced some slackness into the entropy constraints? Or maybe this is a deadend. Or maybe I’m not following right.
Just a note, your notation EP[−logQ[X]] is the cross-entropy H(P,Q) which indeed (Gibbs) is minimised (for given P) when Q=P.
I think this reveals that the maxent problem you post at the start (maximum entropy constrained by cross-entropy not exceeding entropy) is actually fully specified by the constraint, rather than being sensitive to the expression being maximised. So the maxent is irrelevant. Don’t know what downstream implications that has.
For sure you can’t have cross-entropy constrained by both of HP and HQ if they [ETA that is, P and Q, not their entropies] differ at all. Maybe you’d make some headway if you introduced some slackness into the entropy constraints? Or maybe this is a deadend. Or maybe I’m not following right.