Thanks for commenting. I think measuring individual burritos by low information content (i.e. high probability) relative to some recipes doesn’t work. Specifically, this is because the information content is dominated by thermodynamic complexity in the burrito. Under almost any distribution, burritos with low information content will be small, cold, crystalline ones.
Measuring the Kolmogorov complexity of f relative to b seems more promising. This might work if we could set some upper limit on the information content of a distribution containing nano-UFAIs and only allow distributions less (relatively) complex than this limit.
More concretely, we could represent b as a probabilistic program and allow f to be a low-complexity modification of b (measuring complexity using some mutation distribution on probabilistic programs). I think something like this is worth exploring.
Thanks for commenting. I think measuring individual burritos by low information content (i.e. high probability) relative to some recipes doesn’t work. Specifically, this is because the information content is dominated by thermodynamic complexity in the burrito. Under almost any distribution, burritos with low information content will be small, cold, crystalline ones.
Measuring the Kolmogorov complexity of f relative to b seems more promising. This might work if we could set some upper limit on the information content of a distribution containing nano-UFAIs and only allow distributions less (relatively) complex than this limit.
More concretely, we could represent b as a probabilistic program and allow f to be a low-complexity modification of b (measuring complexity using some mutation distribution on probabilistic programs). I think something like this is worth exploring.