Within evolutionary theory, there are two approaches to explaining robustness observed in biological systems. The first is that random search is likely to find basins of robustness simply because such basins occupy significant probability mass. The second approach argues that robustness is selected for by evolution as a response to mutations and environmental perturbations.
In artificial evolutionary search there is an explicit regularization effect over program length/complexity, which thus implements an ockham/MDL like prior ala Solomonoff. That has a strong theoretical foundation for regularization/robustness. In biology every extra DNA base pair costs an extra few ATP of energy for every replication, so there is an energy induced constraint on program length/complexity in biology as well.
I like the systems analogy approach.
In artificial evolutionary search there is an explicit regularization effect over program length/complexity, which thus implements an ockham/MDL like prior ala Solomonoff. That has a strong theoretical foundation for regularization/robustness. In biology every extra DNA base pair costs an extra few ATP of energy for every replication, so there is an energy induced constraint on program length/complexity in biology as well.