Yeah, there’s more to be said about this. If you think of the model as just being the predictions, it doesn’t make sense to imagine handing a person a hypothesis without handing them the ability to figure out what it predicts. As you say, you can instead imagine that a hypothesis is a bayesian network. Or, to be more general, you can imagine it is a program for generating predictions. (A Bayesian network being one particular program.) However, it still doesn’t make very much sense to hand someone a program for generating predictions without having handed them the ability to make predictions! So the confusion persists.
In order to really formalize the distinction I’m making, you have to imagine that you can refer to a hypothesis and put a degree of credence on it without “having it”. So, we still think of a hypothesis as a way of generating predictions, but when I tell you about a hypothesis I’m not necessarily giving you the source code—or maybe I’m giving you the source code, but in a programming language you don’t know how to execute, so you still don’t have enough information to generate predictions.
So, a hypothesis is just a way of generating predictions, but knowing about the hypothesis or knowing some description of it doesn’t necessarily give you access to the hypothesis’ ability to generate predictions.
To clearly illustrate the distinction at play here:
Suppose I have access to a halting oracle which not everyone has access to. I generate predictions about whether a program will halt by consulting the halting oracle. My hypothesis is not “the halting oracle is always right”—that’s a hypothesis which you could also generate predictions from, such as “IF I could access the halting oracle to check it, THEN I would never falsify it”—but rather, my hypothesis is just the halting oracle itself. (There’s a use-mention distinction here.)
If I give you a big bayesian network, you might make different predictions with it than I do because you have different knowledge to give the bayesian network as inputs. However, both of us know how to implement the same input-output relation between observations and predictions; that’s what the bayesian network tells us. However, I can’t give you my procedure for consulting the halting oracle to generate predictions, unless I can give you the halting oracle.
Similarly, if a hypothesis is embodied in a bunch of messy neural connections, we may not be able to convey it perfectly. It’s like we’re all using different programming languages which are so complex that we can’t hope to provide perfect compilers into each other’s dialects.
This point is fairly subtle, since it relies on a distinction between inputs vs subroutines. Am I “observing” the halting oracle to generate my predictions, or am I “calling” it, as one bit of source code may call another? I can think of it both ways. At least it is clear, though, that I’m not “observing” the neural patterns implementing my brain.
Yeah, there’s more to be said about this. If you think of the model as just being the predictions, it doesn’t make sense to imagine handing a person a hypothesis without handing them the ability to figure out what it predicts. As you say, you can instead imagine that a hypothesis is a bayesian network. Or, to be more general, you can imagine it is a program for generating predictions. (A Bayesian network being one particular program.) However, it still doesn’t make very much sense to hand someone a program for generating predictions without having handed them the ability to make predictions! So the confusion persists.
In order to really formalize the distinction I’m making, you have to imagine that you can refer to a hypothesis and put a degree of credence on it without “having it”. So, we still think of a hypothesis as a way of generating predictions, but when I tell you about a hypothesis I’m not necessarily giving you the source code—or maybe I’m giving you the source code, but in a programming language you don’t know how to execute, so you still don’t have enough information to generate predictions.
So, a hypothesis is just a way of generating predictions, but knowing about the hypothesis or knowing some description of it doesn’t necessarily give you access to the hypothesis’ ability to generate predictions.
To clearly illustrate the distinction at play here:
Suppose I have access to a halting oracle which not everyone has access to. I generate predictions about whether a program will halt by consulting the halting oracle. My hypothesis is not “the halting oracle is always right”—that’s a hypothesis which you could also generate predictions from, such as “IF I could access the halting oracle to check it, THEN I would never falsify it”—but rather, my hypothesis is just the halting oracle itself. (There’s a use-mention distinction here.)
If I give you a big bayesian network, you might make different predictions with it than I do because you have different knowledge to give the bayesian network as inputs. However, both of us know how to implement the same input-output relation between observations and predictions; that’s what the bayesian network tells us. However, I can’t give you my procedure for consulting the halting oracle to generate predictions, unless I can give you the halting oracle.
Similarly, if a hypothesis is embodied in a bunch of messy neural connections, we may not be able to convey it perfectly. It’s like we’re all using different programming languages which are so complex that we can’t hope to provide perfect compilers into each other’s dialects.
This point is fairly subtle, since it relies on a distinction between inputs vs subroutines. Am I “observing” the halting oracle to generate my predictions, or am I “calling” it, as one bit of source code may call another? I can think of it both ways. At least it is clear, though, that I’m not “observing” the neural patterns implementing my brain.