Thanks for this magnificent post. My only concern is that the point seems slightly overstated when you write: “All learning is induction, and all induction takes place through inductive bias.” I wish this had been phrased slightly differently. The definition of learning seems a bit narrow. Is there no such thing as deductive learning? But even considering only the realm of inductive learning (based on observation), let’s assume I see a swan for the first time, and the swan is white. Wouldn’t it be correct to say that I’ve learned that at least one swan is white? (This may be slow learning, given the context, but wouldn’t it still be learning?) And isn’t the “inductive bias” in this case so minimal that it’s not really properly called “bias” at all, since the assumption cannot be false?
Thanks for this magnificent post. My only concern is that the point seems slightly overstated when you write: “All learning is induction, and all induction takes place through inductive bias.” I wish this had been phrased slightly differently. The definition of learning seems a bit narrow. Is there no such thing as deductive learning? But even considering only the realm of inductive learning (based on observation), let’s assume I see a swan for the first time, and the swan is white. Wouldn’t it be correct to say that I’ve learned that at least one swan is white? (This may be slow learning, given the context, but wouldn’t it still be learning?) And isn’t the “inductive bias” in this case so minimal that it’s not really properly called “bias” at all, since the assumption cannot be false?