Obvious / informal connection between SLT and information bottleneck theory:SLT says: more degeneracies → better generalization IB says: less mutual information b/w input and representation → better generalization the more degeneracies a function has, the less information it can preserve.
Obvious / informal connection between SLT and information bottleneck theory:
SLT says: more degeneracies → better generalization
IB says: less mutual information b/w input and representation → better generalization
the more degeneracies a function has, the less information it can preserve.