Thanks for the comment! The way I think about it, there are several ways of thinking about RG in terms of different ‘energy’ analogues in NNs, and each is likely tied to a different framing in terms of ‘UV’ and ‘IR’.
For example, during training, you start with a simplified (IR like) description of the dataset that flows to a richer representation, adding finer grained structure capable of generalizing (UV).
During inference, I agree that you can describe this process as UV → IR, as each layer is a progressively coarser representation as the features that are irrelevant for a certain task (like classification) are ‘integrated out’ to yield a usefully abstract simplification. However, you can also think of inference in terms of ‘feature refinement’, where each layer becomes progressively more structured, able to pick up on finer or more abstract details. This ultimately depends on how you think of ‘scale’ along the RG flow.
Thanks for the comment! The way I think about it, there are several ways of thinking about RG in terms of different ‘energy’ analogues in NNs, and each is likely tied to a different framing in terms of ‘UV’ and ‘IR’.
For example, during training, you start with a simplified (IR like) description of the dataset that flows to a richer representation, adding finer grained structure capable of generalizing (UV).
During inference, I agree that you can describe this process as UV → IR, as each layer is a progressively coarser representation as the features that are irrelevant for a certain task (like classification) are ‘integrated out’ to yield a usefully abstract simplification. However, you can also think of inference in terms of ‘feature refinement’, where each layer becomes progressively more structured, able to pick up on finer or more abstract details. This ultimately depends on how you think of ‘scale’ along the RG flow.