Dalcy comments on The Hessian rank bounds the learning coefficient

Dalcy 31 Mar 2025 8:36 UTC
4 points
0
Where in the literature can I find the proof of the lower bound?
- Dalcy 2 Aug 2025 19:57 UTC
  8 points
  0
  Parent
  Found a proof sketch here (App. D.3), couldn’t it find elsewhere in canonical SLT references eg gray book. Idea seems simple:
  1. (We’ll prove it for the local RLCT because the splitting lemma most naturally applies when dealing with local diffeomorphisms—but if you’re interested in the statement for the global RLCT, then since RLCT is min over local RLCT (3.2.2), just work with the local RLCT at the minimizing point and the rest of the argument follows)
  2. Local RLCT is invariant under local diffeomorphism, easy to prove by the volume scaling formula (check exercise 17).
  3. Apply the splitting lemma (elementary proof in Gilmore 3.4) to locally express the function as a quadratic in the nondegenerate plus the rest.
  4. Use Remark 7.2.3 in the gray book which says that if $K (w_{a}, w_{b}) = K_{a} (w_{a}) + K_{b} (w_{b})$ and $φ (w_{a}, w_{b}) = φ_{a} (w_{a}) φ_{b} (w_{b})$ (which the change of variable in the splitting lemma satisfies), then $λ = λ_{a} + λ_{b}$ (also easy to prove with the volume scaling formula).
  5. So $λ$ is lower-bounded by the RLCT in the quadratic part, which is $\frac{d_{1}}{2}$ (again easy to prove with the volume scaling formula, check pg 17 here)