What do you think about making burritos that have low information content given good burrito recipes (or sampling from a distribution f that has low information relative to the learned distribution b)? Fancy burritos seem a lot simpler to describe than nanobot-containing burritos, given the machinery to describe mediocre burritos.
Thanks for commenting. I think measuring individual burritos by low information content (i.e. high probability) relative to some recipes doesn’t work. Specifically, this is because the information content is dominated by thermodynamic complexity in the burrito. Under almost any distribution, burritos with low information content will be small, cold, crystalline ones.
Measuring the Kolmogorov complexity of f relative to b seems more promising. This might work if we could set some upper limit on the information content of a distribution containing nano-UFAIs and only allow distributions less (relatively) complex than this limit.
More concretely, we could represent b as a probabilistic program and allow f to be a low-complexity modification of b (measuring complexity using some mutation distribution on probabilistic programs). I think something like this is worth exploring.
What do you think about making burritos that have low information content given good burrito recipes (or sampling from a distribution f that has low information relative to the learned distribution b)? Fancy burritos seem a lot simpler to describe than nanobot-containing burritos, given the machinery to describe mediocre burritos.
Thanks for commenting. I think measuring individual burritos by low information content (i.e. high probability) relative to some recipes doesn’t work. Specifically, this is because the information content is dominated by thermodynamic complexity in the burrito. Under almost any distribution, burritos with low information content will be small, cold, crystalline ones.
Measuring the Kolmogorov complexity of f relative to b seems more promising. This might work if we could set some upper limit on the information content of a distribution containing nano-UFAIs and only allow distributions less (relatively) complex than this limit.
More concretely, we could represent b as a probabilistic program and allow f to be a low-complexity modification of b (measuring complexity using some mutation distribution on probabilistic programs). I think something like this is worth exploring.