The ultimate chess playing Giant Look-Up Table is huge, but it has small Kolmogorov complexity.
(Or rather, it depends on some technical details. Sometimes there is more than one winning move; which one will the table generator choose? If there is a simple rule, such as “choose the move that comes first in the alphabet”, then the Kolmogorov complexity of the table is low. But if the rule is “choose a random move”, then the Kolmogorov complexity of the table is high, because it encodes not only the algorithm that created the table, but also all the random numbers that were used.)
I am not sure what the experts mean when they say “interpretability”, but I would expect something like small Kolmogorov complexity to be involved.
The ultimate chess playing Giant Look-Up Table is huge, but it has small Kolmogorov complexity.
(Or rather, it depends on some technical details. Sometimes there is more than one winning move; which one will the table generator choose? If there is a simple rule, such as “choose the move that comes first in the alphabet”, then the Kolmogorov complexity of the table is low. But if the rule is “choose a random move”, then the Kolmogorov complexity of the table is high, because it encodes not only the algorithm that created the table, but also all the random numbers that were used.)
I am not sure what the experts mean when they say “interpretability”, but I would expect something like small Kolmogorov complexity to be involved.
Yes, the point of this post is that low Kolmogorov complexity doesn’t automatically yield high interpretability.