I’m not so sure it is addressed by reframing in probabilistic terms. At least for now, I’m convinced of a version of Popper’s argument against probabilistic reasoning (admittedly, probably an over-simplified version
That argument ends with the conclusion that evidence can support any number of hypotheses. But everybody knows that. Any Bayesian or rationalists would say that’s what you need simplicity criteria for.
Sure, but that adds the additional assumption about simplicity, and it concedes that evidence doesn’t weigh more in one generalization’s favor over another. Bayesianism requires this extra axiom, which ironically makes it less simple (unless you want to reason to it from Bayes’ Theorem, but that ignores the fact that P(E|G) should be 1). In contrast, simplicity is desired on a Popperian account because it makes hypotheses easier to test; clear simple predictions are usually easier to falsify.
I think Popper’s point is that induction was never needed in the first place. Knowledge grows through the process I mentioned, and we don’t need to assume any inductive trick that necessarily (even probabilistically) gets us to general laws. We can accept that knowledge is fundamentally trial-and-error or guess-and-check, and the phenomenon of knowledge-production loses nothing.
Sure, but that adds the additional assumption about simplicity
Why is that a problem? There is still a form of probabilistic inductive reasoning that works.
and it concedes that evidence doesn’t weigh more in one generalization’s favor over another.
Why is that a problem? Everybody uses simplicity criteria, so no one has the problem.
Bayesianism requires this extra axiom, which ironically makes it less simple
Less simple than...something that doesn’t work. The point of simplicity is to get the simplest working theory.
simplicity is desired on a Popperian account because it makes hypotheses easier to test; clear simple predictions are usually easier to falsify.
Ok. So they both use simplicity, and for different reasons. That isn’t telling me induction doesn’t work.
I think Popper’s point is that induction was never needed in the first place.
Induction is useful. There is value in knowing, even without certainty, what will happen next, even without having the explanatory knowledge of why it will happen. There is more value in having the explanatory knowledge as well, but that doesn’t mean there is zero value in non-explanatory prediction.
It’s significant that even simple organisms use induction..they expect the food that made them sick before to make them sick again.. That tells you that induction is useful...and that it is not difficult to implement.
In any case “induction isn’t needed” is a different claim to “induction doesn’t work”.
Knowledge grows through the process I mentioned, and we don’t need to assume any inductive trick that necessarily (even probablistically) gets us to general laws. We can accept that knowledge is fundamentally trial-and-error or
Bayes, correctly understood, isn’t something different to trial and error. It doesn’t give you a mechanism to generate hypotheses, so you have to conjecture them. And it does give you a mechanism to falsify them (although it does so incrementally, not all at once like naive Popperism). It’s a mistake to suppose that just because you have two different” schools”, they can’t agree on a single point.
Popperism, correctly understood, isn’t just trial and error, either.
Why is that a problem? There is still a form of probabilistic inductive reasoning that works.
I’m just articulating Popper’s views to the best of my ability, and he did not believe in probabilistic induction. He explains what we call ‘inductive reasoning’ as mere conjecture (perhaps calibrating for some psychological biases), but that’s not actually reasoning (at least to Popper). The reason assuming simplicity as a fundamental criterion is a ‘problem’ as you said, is because you don’t need to. As I mentioned, Popper can account for the desirability of simplicity without assuming it as an axiom.
That isn’t telling me induction doesn’t work.
That wasn’t the argument against induction. It was an accounting for simplicity without assuming it.
Induction is useful...In any case “induction isn’t needed” is a different claim to “induction doesn’t work”.
Again, just through a Popperian lens: prediction is useful, even without explanation. However, prediction on its own is not knowledge. For Popper, induction was just never a thing to begin with. I think my point in saying “induction was never needed in the first place” was to emphasize that we can still account for knowledge production without induction. I agree it’s different from saying “induction doesn’t work,” but if the logic of induction is not warranted (perhaps by showing it doesn’t mathematically work), then induction isn’t a thing.
It’s a mistake to suppose that just because you have two different” schools”, they can’t agree on a single point.
Totally agree, but without induction, guess-and-check is perhaps the most primitive way of describing Popperianism. To be a little triggering, induction would change it to semi-prescient guess-and-check. Regardless, I agree this is a bit of an oversimplification.
I’m just articulating Popper’s views to the best of my ability
I’m interested in whether his views are supported by sound arguments.
He explains what we call ‘inductive reasoning’ as mere conjecture
But induction can be performed by organisms and software too simple to form conjectures.
, but that’s not actually reasoning (at least to Popper).
Maybe that’s a True Scotsman argument.
The reason assuming simplicity as a fundamental criterion is a ‘problem’ as you said, is because you don’t need to.
You don’t need to assume it because you can argue for it methodologicaly.
There are more complex conjectures than simple ones. So if you conjecture something complex, it is less likely to be the right conjecture. Also, you have only a finite amount of time to consider conjectures, so you can’t start at the end an infinite list..But you can start with th the simplest conjecture. That’s f course, that’s roughly his Solomonoff induction works.
The argument is in favour of relative simplicity: it doesn’t assume that the universe has any absolute level of simplicity.
As I mentioned, Popper can account for the desirability of simplicity without assuming it as an axiom.
I am not sure what “assuming it as an axiom” means. I can argue for simplicity on methodological grounds. Poppers argument for simplicity is also methodological , so what am I doing wrong that he is doing right?
However, prediction on its own is not knowledge
The also sounds like a true Scotsman.
For Popper, induction was just never a thing to begin with.
Unless it’s conjecture.
we can still account for knowledge production without induction.
We can’t account for the production of all kinds of knowledge without induction, because it produces one of the kinds.
but if the *logic *of induction is not warranted (perhaps by showing it doesn’t mathematically work), then induction isn’t a thing.
But I believe it is a thing, because it can be demonstrated directly , and because thrreare not any sound and valid argument against it that I have seen.The
(What is the best argument against it, and why not quite it directly?)
But induction can be performed by organisms and software too simple to form conjectures.
I’m actually not as well informed on how reasoning operates in other organisms, but if you are allowing for primitive structures that enable some kind of proto-inductive reasoning, then I have no idea why you wouldn’t also allow for primitive structures that enable proto-conjecture. If there’s a distinction between the two, then surely conjecture would operate on even more primitive mechanics than inductive reasoning. Automated systems are mostly doing optimization, which is sort of in a totally different camp, but I’d allow for the possibility that something like Attention in LLMs is simulating some kind of intuition. Still, that’s a total guess on my part.
Maybe that’s a True Scotsman argument...This* also sounds like a true Scotsman.
Neither were No-True-Scotsman’s, since I defined what Popper meant by “reasoning” and “knowledge” in my very first post, so these exclusions were not merely arbitrary.
You don’t need to assume it because you can argue for it methodologically*.
I’m certainly interested in diving into this more, since I find the math fascinating. Nonetheless, in my 3rd post when I said:
...unless you want to reason to it...
I was giving you the option to accept it axiomatically vs. deriving it, and perhaps it’s on me that I interpreted your following responses as taking the former position. However, if you derive it, you must derive it utilizing probabilistic reasoning (as it sounds like you do), which runs into the issue I mentioned; namely P(E|H)=1, because that’s what evidence means. Evidence is a logical consequence of the hypothesis, so H⟹E, or if you want to view them as sets of possible worlds, H⊆E. This immediately implies P(E|H)=1. This result ruins the desired Bayesian updating for induction, which was the whole reason why you mentioned simplicity, which was the reason I assumed it was your axiom.
However, a better critique would be the very notion of probability as uncertainty. (This argument is my own) Probability is merely normalized measure defined on some σ-algebra (a definition of measurable sets). It can be used to model measurement errors, frequencies of outcomes, perhaps the number of indiscernible symmetries of a system, and some uncertainties if it is well understood that the model is accurate. However, to model all uncertainty as purely probabilistic as if that is the one true philosophic interpretation of probability I find overly presumptuous. What makes Bayesians think the set of all subjective uncertainties can be modeled via some σ-algebra? I’ll grant we don’t need to know the density function (that’s the point of Bayesian statistics), but we should at least know the σ-algebra so we can know to what to apply probability.
Unless it’s conjecture.
Again, see the definition of reasoning I mentioned above.
We can’t account for the production of all kinds of knowledge without induction, because it produces one of the kinds.
Which kind of knowledge does it produce that is not produced deductively?
But I believe it is a thing, because it can be demonstrated directly , and because there* are not any sound and valid arguments* against it that I have seen.
Is there any demonstrations of induction that are not merely conjecture? I would say the argument above is valid, and also from the link I shared. If you disagree, point to the logical error.
What is the best argument against it, and why not quote* it directly?
Frankly, because I don’t know it. I don’t even know the technical statement, let alone its proof. What I presented here is my understanding, and the link I provided is a better articulation of the extent of my understanding. The Popper-Miller Theorem is notoriously difficult to understand and frequently misconstrued, so I chose to point to one that better represented my understanding. Feel free to look it up, but since I have enough expertise in logic and mathematics to be confident I could understand it, as well as an interest to eventually understand it, I’m not going to be particularly moved by any LW criticisms of the theorem until I do so.
I’m actually not as well informed on how reasoning operates in other organisms, but if you are allowing for primitive structures that enable some kind of proto-inductive reasoning, then I have no idea why you wouldn’t also allow for primitive structures that enable proto-conjecture.
Remember, I’m talking about algorithms as well, and simple ones. Much simpler than current LLMs, the kind that could be constructed decades ago.
If you write an algorithm, it’s a white box to you—there’s no mystery how it works. And you can write an algorithm that is hard coded to expect patterns to repeat, that has the ultimate inductive bias. That is simpler than writing something that conjectures repeating patterns. For instance, with 1969s technology, you can write an inductor that figures out that q’s are likely to be followed by u’s in english text.
Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up.
Modern A Is at capable of inferring patterns that aren’t hard coded, and they are much more complex than 1960′s GOFAI This is not a coincidence.
If there’s a distinction between the two, then surely conjecture would operate on even more primitive mechanics than inductive reasoning.
You need more of an argument than the word “surely” .
Automated systems are mostly doing optimization, which is sort of in a totally different camp, but I’d allow for the possibility that something like Attention in LLMs is simulating some kind of intuition.
Did you mean “induction” ?
However, if you derive it, you must derive it utilizing probabilistic reasoning (as it sounds like you do), which runs into the issue I mentioned; namely P(E|H)=1 ,
Not under probabilistic reasoning! The hypothesis that text is in English implies a high probability that q’sill be followed by u’s, but not a certainty, there are some exceptions.
However, to model all uncertainty as purely probabilistic as if that is the one true philosophic interpretation of probability I find overly presumptuous
So what is non probabilistic uncertainy? Something that doesn’t follow the laws of probability?
But that’s an argument for something general than probability … but the claim that probability theory can found inductionism doesn’t depend on probability being completely general.
(And I said probabilistic reasoning, not Bayes. There are prop!e within Bayes is the one true probability theory and/or a completely general epistemology … But I’m not one of them)
Neither were No-True-Scotsman’s, since I defined what Popper meant by “reasoning” and “knowledge” in my very first post, so these exclusions were not merely arbitrary.
...on your part. The problem is his true scotsmanning. He has these claims that “knowledge is..” , but they are based on defining knowledge, not on making discoveries about the world.
We can’t account for the production of all kinds of knowledge without induction, because it produces one of the kinds.
Which kind of knowledge does it produce that is not produced deductively
Non explanatory predictive knowledge.
Is there any demonstrations of induction that are not merely conjecture?
If you mean inductions in an agent occurring without that agent itself making a conjecture ,then yes, I have argued that.
Frankly, because I don’t know it.
So why are you so sure the claim is true?
The Popper-Miller Theorem is notoriously difficult to understand and frequently misconstrued
That is simpler than writing something that conjectures repeating patterns...
Is it simpler? Based on your description, it doesn’t sound simpler. In fact, if you asked me to write a program to conjecture repeating patterns, I would probably end up writing exactly what you would describe as prediction. From what I can tell, this is a distinction without a difference.
You need more of an argument than the word “surely” .
I actually don’t need an argument because you are the one claiming a distinction. My claim was an expression of my intuition given your hypothetical, which is meant to query you for said distinction.
Did you mean “induction” ?
Nope.
Not under probabilistic reasoning! The hypothesis that text is in English implies a high probability that q’s will* be followed by u’s, but not a certainty, there are some exceptions.
Here you are making a slight equivocation between different versions of E and different interpretations of probability:
H=The text is English,
E=q’s are always followed by u’s in the text.
That’s different from
H=The text is English,
E′=q’s are followed by u’s in the text with high probability.
In the former case, E is not a logical consequence of H, and so P(E|H) could be less than 1 (though in this sense, probability would not be interpreted as a credence, but as a frequency of letter usage). In the latter case, E’ is a logical consequence of H, so interpreted as credence P(E′|H)=1. In this case, E’ itself is a probabilistic statement (letter frequency) which can be true or false, and it is guaranteed to be true if H is true.
So what is non probabilistic uncertainy? Something that doesn’t follow the laws of probability?
Uncertainty is a subjective feeling, and it still needs to be demonstrated that this feeling can be modeled by probability. I would be careful not to shift the burden of proof. It is the job of the Bayesian to prove we can model all uncertainty this way, not the job of the skeptic to disprove it. As the skeptic, I’ll be careful not to strawman, but if probability really does ground epistemology, then I don’t think it’s a stretch to characterize this as assuming all boolean statements can be taken as inputs to a probability measure, representing our credence. If that’s not the case, I’d be open to an alternative interpretation.
...on your part. The problem is his true scotsmanning. He has these claims that “knowledge is..” , but they are based on defining knowledge, not on making discoveries about the world.
As I mentioned before, bickering over definitions was never Popper’s intention. He was far more interested in explaining the phenomenon of knowledge, and from these pursuits he established his definitions. That’s different from editing one’s definition to exclude arbitrary counterexamples (a No-True-Scotsman). I think we can all agree that the definitions aren’t the point; they are just the facilitators of the communication of phenomena. It’s fine if we disagree on definitions, as long as we agree on the underlying phenomenon.
Non explanatory predictive knowledge.
Given what I just said, perhaps it’s better to rephrase my question: what phenomenon remains unaccounted for without a distinction between inductive reasoning and conjecture? The phenomenon of non-explanatory prediction is classified as a type of conjecture in Popper’s terms.
If you mean inductions in an agent occurring without that against itself making making a conjecture ,then yes, I have argued that.
I’m not sure what you just said.
So why are you so sure the claim is true?
At the very beginning, I said I lean critical rationalist. I am sure the Popper-Miller theorem is valid given that it is not called a conjecture or a blunder, and anecdotally I have heard mathematicians say it is mathematically obvious (though philosophically confusing), but I am not sure in the sense of understanding it and checking the proof myself. We believe all sorts of things without first-hand corroboration.
and not widely accepted!
Philosophers tend to find the math confusing, and mathematicians tend to find the philosophy confusing. This is not a recipe for acceptance corresponding with corroboration, so that is just an appeal to ignorance.
a) code to generate conjectures
b) code to test the conjectures
c) code to reject bad conjectures , and go back a)
Whereas I only need to write b)
I actually don’t need an argument because you are the one claiming a distinction
Here’s the argument supporting the claim, again:-
“Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up”
In this case, E’ itself is a probabilistic statement (letter frequency) which can be true or false, and it is guaranteed to be true if H is true.
Why does that matter?
Uncertainty is a subjective feeling, and it still needs to be demonstrated that this feeling can be modeled by probability
It sometimes can, since probability sometimes works. Maybe it sometimes d
oesn’t, but I sent see how that results in a sweeping deposit of Induction.
It is the job of the Bayesian to prove we can model all uncertainty
Im not defending Bayesianism in that sense, as I said.
As I mentioned before, bickering over definitions was never Popper’s intention
Maybe smuggling in definitions without inconvenient bickering was the intention...you are not automatically on the epistemological high ground when you refuse to engage in “semantics”
Given what I just said, perhaps it’s better to rephrase my question: what phenomenon remains unaccounted for without a distinction between inductive reasoning and conjecture
The ability of agents too simple to form conjectures to nonetheless perform inductive reasoning.
. I am sure the Popper-Miller theorem is valid given that it is not called a conjecture or a blunder
By its authors. But a number of criticisms and counterarguments have been published, eg:-
Perhaps you know that the Popper-Miller argument has a serious logical flaw identified by Richard Jeffrey in his 1983 book “The Logic
of Decision” when it was first published in their 1983 letter to Nature ? Popper and Miller seemed to just ignore the flaw and republish it in PTRSL four years later.
Their argument can be summarised as follows. They seek to establish whether a hypothesis H acquires inductive support under Bayesian
theory by the evidence E. H can be expressed logically as (H or not E) and (H or E). H or not E is equivalent to the statement “H is true given E is true” or simply “H if E”. Now (H or E) is implied by E trivially so they focus on how E supports (H if E). Now Pr ((H if E) given E) is
clearly ⇐ Pr(H if E) so that (H if E) never is
incrementally confirmed by E.
The flaw lies in their claim that “H if E” is that part of H over
and above E (i.e. H and not E). It is in fact all of H as well as all of that which is not E.
By the way, an argument can be valid mathematically , but still fail to represent the real world. Conveniently, Vasrani’s argument has that property.
Philosophers tend to find the math confusing, and mathematicians tend to find the philosophy confusing.
If Popper and Miller have both competencies , others could as well.,
...but I cant* see how that results in a sweeping deposit of Induction.
It doesn’t, but it was at least convincing to me that probabilistic reasoning is much more vague than it makes itself out to be.
I’m not defending Bayesianism in that sense, as I said.
Sounds good.
...you are not automatically on the epistemological high ground when you refuse to engage in “semantics”
Agreed, but choosing to focus on the referent rather than the sense while acknowledging the different senses is the ‘high ground’ as you said, and it is an explicit engaging in semantics. I’m happy to discard or adopt terms if they are shown to be obfuscating or useful respectively.
“Whereas I only need to write b)...
″...Here’s the argument supporting the claim, again:-
‘Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up’...”
″...The ability of agents too simple to form conjectures to nonetheless perform inductive reasoning...”
Perfect, I’m lumping these together because I’m realizing this is the crux and perhaps you can consolidate further. I apologize if I didn’t adequately respond to your other instantiations of these.
For your a)b)c) program, I was only talking about conjectures in that thread, so I would only need to write (a). Is (a) necessarily more complicated than whatever mechanism you have for induction? Also, for me (b) only consists of deductive falsifications, so what you call “induction” would still be part of (a).
For your white box example, it’s not clear to me how initialized expectations are not the same as conjectural dispositions.
For simple agential models which cannot conjecture but still perform inductive reasoning, I’m curious what mechanisms you think are sufficient for conjecture and what mechanisms are necessary for induction? Obviously, for very simple agents, “conjecturing” and “reasoning” aren’t exactly writing down logical statements in English. We’re probably talking about encoding information somehow? Inductive bias, like how ML systems work?
But a number of criticisms and counterarguments have been published
Yeah, and I’ll definitely be looking into those as well. I look forward to it!
By the way, an argument can be valid mathematically , but still fail to represent the real world. Conveniently, Vasrani’s argument has that property.
Totally agree with the first part.
If Popper and Miller have both competencies , others could as well.
Definitely, and I hope to be one, but the discourse around it does not inspire confidence.
...but I cant* see how that results in a sweeping deposit of Induction.
It doesn’t, but it was at least convincing to me that probabilistic reasoning is much more vague than it makes itself out to be.
Why is that interesting to me? AFAIC ,the debate is about whether induction works. So I’m not interested in general point scoring against Bayes or probability
For your a)b)c) program, I was only talking about conjectures in that thread, so I would only need to write (a).
forming conjectures without any attempt to refute or support them is not knowledge generation.
Also, for me (b) only consists of deductive falsifications, so what you call “induction” would still be part of (a).
I’m stipulating that b) is a simple inductor.
Obviously, for very simple agents, “conjecturing” and “reasoning” aren’t exactly writing down logical statements in English. We’re probably talking about encoding information somehow? Inductive bias, like how ML systems work?
No, it’s just doing something in a hard coded way. Not generating an English level description of what to do, interpreting it, and executing it.
Because either you are not updating credence (which I have no objection to), or you can’t distinguish between hypotheses without assuming simplicity as an axiom (which, feel free to do so, but I already argued it doesn’t need to be assumed). But I think this train of thought seems less important than the necessity of induction discussion in the other threads.
Why is that interesting to me?
It doesn’t need to be. I just found it more compelling.
forming conjectures without any attempt to refute or support them is not knowledge generation.
Totally agree. So I think we may have talked past each other a bit because I was only comparing induction to conjecture, not the full knowledge-generation process. Sure (b) alone is simpler than (a), (b), and (c) collectively, but that’s not what I was arguing against.
I’m stipulating that b) is a simple inductor.
Okay, well that’s a bit of a bedrock of disagreement then.
No, it’s just doing something in a hard coded way. Not generating an English level description of what to do, interpreting it, and executing it.
Sure, so what is your sufficient condition for conjecture to be present, and what is your necessary condition for induction to be present?
can’t distinguish between hypotheses without assuming simplicity as an axiom (which, feel free to do so, but I already argued it doesn’t need to be assumed).
So have I.:-
There are more complex conjectures than simple ones. So if you conjecture something complex, it is less likely to be the right conjecture. Also, you have only a finite amount of time to consider conjectures, so you can’t start at the end an infinite list..But you can start with th the simplest conjecture. Of course, that’s roughly how Solomonoff induction works.
(Also, it is completely unclear why “having to assume simplicity” amounts to “not working”. You could argue, as Vasrani does that Bayes without simplicity doesn’t work: I have argued that no real Bayesian ignores simplicity).
but that’s not what I was arguing against
Why not? An aircraft without wing s or engine is sim ple, but it can’t fly.
Okay, well that’s a bit of a bedrock of disagreement then
Because you think I was stipulating something else? Because you think there are no simple inductors?
Sure, so what is your sufficient condition for conjecture to be present, and what is your necessary condition for induction to be present
You can tell that a algorithm is making predictions on a black box basis , and you can tell it’s an inductor if it does immediately on boot up.
A conjecture-and-refutation machine has to be complex enough to form high level representations, and make inferences from them.
I think in each of these threads, we’ve started to go in circles, so if it’s any consolation I’m interested in following your future posts, and if I post anything in the future I would be interested to see your critiques.
That argument ends with the conclusion that evidence can support any number of hypotheses. But everybody knows that. Any Bayesian or rationalists would say that’s what you need simplicity criteria for.
Sure, but that adds the additional assumption about simplicity, and it concedes that evidence doesn’t weigh more in one generalization’s favor over another. Bayesianism requires this extra axiom, which ironically makes it less simple (unless you want to reason to it from Bayes’ Theorem, but that ignores the fact that P(E|G) should be 1). In contrast, simplicity is desired on a Popperian account because it makes hypotheses easier to test; clear simple predictions are usually easier to falsify.
I think Popper’s point is that induction was never needed in the first place. Knowledge grows through the process I mentioned, and we don’t need to assume any inductive trick that necessarily (even probabilistically) gets us to general laws. We can accept that knowledge is fundamentally trial-and-error or guess-and-check, and the phenomenon of knowledge-production loses nothing.
Why is that a problem? There is still a form of probabilistic inductive reasoning that works.
Why is that a problem? Everybody uses simplicity criteria, so no one has the problem.
Less simple than...something that doesn’t work. The point of simplicity is to get the simplest working theory.
Ok. So they both use simplicity, and for different reasons. That isn’t telling me induction doesn’t work.
Induction is useful. There is value in knowing, even without certainty, what will happen next, even without having the explanatory knowledge of why it will happen. There is more value in having the explanatory knowledge as well, but that doesn’t mean there is zero value in non-explanatory prediction.
It’s significant that even simple organisms use induction..they expect the food that made them sick before to make them sick again.. That tells you that induction is useful...and that it is not difficult to implement.
In any case “induction isn’t needed” is a different claim to “induction doesn’t work”.
Bayes, correctly understood, isn’t something different to trial and error. It doesn’t give you a mechanism to generate hypotheses, so you have to conjecture them. And it does give you a mechanism to falsify them (although it does so incrementally, not all at once like naive Popperism). It’s a mistake to suppose that just because you have two different” schools”, they can’t agree on a single point.
Popperism, correctly understood, isn’t just trial and error, either.
I’m just articulating Popper’s views to the best of my ability, and he did not believe in probabilistic induction. He explains what we call ‘inductive reasoning’ as mere conjecture (perhaps calibrating for some psychological biases), but that’s not actually reasoning (at least to Popper). The reason assuming simplicity as a fundamental criterion is a ‘problem’ as you said, is because you don’t need to. As I mentioned, Popper can account for the desirability of simplicity without assuming it as an axiom.
That wasn’t the argument against induction. It was an accounting for simplicity without assuming it.
Again, just through a Popperian lens: prediction is useful, even without explanation. However, prediction on its own is not knowledge. For Popper, induction was just never a thing to begin with. I think my point in saying “induction was never needed in the first place” was to emphasize that we can still account for knowledge production without induction. I agree it’s different from saying “induction doesn’t work,” but if the logic of induction is not warranted (perhaps by showing it doesn’t mathematically work), then induction isn’t a thing.
Totally agree, but without induction, guess-and-check is perhaps the most primitive way of describing Popperianism. To be a little triggering, induction would change it to semi-prescient guess-and-check. Regardless, I agree this is a bit of an oversimplification.
I’m interested in whether his views are supported by sound arguments.
But induction can be performed by organisms and software too simple to form conjectures.
Maybe that’s a True Scotsman argument.
You don’t need to assume it because you can argue for it methodologicaly.
There are more complex conjectures than simple ones. So if you conjecture something complex, it is less likely to be the right conjecture. Also, you have only a finite amount of time to consider conjectures, so you can’t start at the end an infinite list..But you can start with th the simplest conjecture. That’s f course, that’s roughly his Solomonoff induction works.
The argument is in favour of relative simplicity: it doesn’t assume that the universe has any absolute level of simplicity.
I am not sure what “assuming it as an axiom” means. I can argue for simplicity on methodological grounds. Poppers argument for simplicity is also methodological , so what am I doing wrong that he is doing right?
The also sounds like a true Scotsman.
Unless it’s conjecture.
We can’t account for the production of all kinds of knowledge without induction, because it produces one of the kinds.
But I believe it is a thing, because it can be demonstrated directly , and because thrreare not any sound and valid argument against it that I have seen.The
(What is the best argument against it, and why not quite it directly?)
I’m actually not as well informed on how reasoning operates in other organisms, but if you are allowing for primitive structures that enable some kind of proto-inductive reasoning, then I have no idea why you wouldn’t also allow for primitive structures that enable proto-conjecture. If there’s a distinction between the two, then surely conjecture would operate on even more primitive mechanics than inductive reasoning. Automated systems are mostly doing optimization, which is sort of in a totally different camp, but I’d allow for the possibility that something like Attention in LLMs is simulating some kind of intuition. Still, that’s a total guess on my part.
Neither were No-True-Scotsman’s, since I defined what Popper meant by “reasoning” and “knowledge” in my very first post, so these exclusions were not merely arbitrary.
I’m certainly interested in diving into this more, since I find the math fascinating. Nonetheless, in my 3rd post when I said:
I was giving you the option to accept it axiomatically vs. deriving it, and perhaps it’s on me that I interpreted your following responses as taking the former position. However, if you derive it, you must derive it utilizing probabilistic reasoning (as it sounds like you do), which runs into the issue I mentioned; namely P(E|H)=1, because that’s what evidence means. Evidence is a logical consequence of the hypothesis, so H⟹E, or if you want to view them as sets of possible worlds, H⊆E. This immediately implies P(E|H)=1. This result ruins the desired Bayesian updating for induction, which was the whole reason why you mentioned simplicity, which was the reason I assumed it was your axiom.
However, a better critique would be the very notion of probability as uncertainty. (This argument is my own) Probability is merely normalized measure defined on some σ-algebra (a definition of measurable sets). It can be used to model measurement errors, frequencies of outcomes, perhaps the number of indiscernible symmetries of a system, and some uncertainties if it is well understood that the model is accurate. However, to model all uncertainty as purely probabilistic as if that is the one true philosophic interpretation of probability I find overly presumptuous. What makes Bayesians think the set of all subjective uncertainties can be modeled via some σ-algebra? I’ll grant we don’t need to know the density function (that’s the point of Bayesian statistics), but we should at least know the σ-algebra so we can know to what to apply probability.
Again, see the definition of reasoning I mentioned above.
Which kind of knowledge does it produce that is not produced deductively?
Is there any demonstrations of induction that are not merely conjecture? I would say the argument above is valid, and also from the link I shared. If you disagree, point to the logical error.
Frankly, because I don’t know it. I don’t even know the technical statement, let alone its proof. What I presented here is my understanding, and the link I provided is a better articulation of the extent of my understanding. The Popper-Miller Theorem is notoriously difficult to understand and frequently misconstrued, so I chose to point to one that better represented my understanding. Feel free to look it up, but since I have enough expertise in logic and mathematics to be confident I could understand it, as well as an interest to eventually understand it, I’m not going to be particularly moved by any LW criticisms of the theorem until I do so.
Remember, I’m talking about algorithms as well, and simple ones. Much simpler than current LLMs, the kind that could be constructed decades ago.
If you write an algorithm, it’s a white box to you—there’s no mystery how it works. And you can write an algorithm that is hard coded to expect patterns to repeat, that has the ultimate inductive bias. That is simpler than writing something that conjectures repeating patterns. For instance, with 1969s technology, you can write an inductor that figures out that q’s are likely to be followed by u’s in english text.
Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up.
Modern A Is at capable of inferring patterns that aren’t hard coded, and they are much more complex than 1960′s GOFAI This is not a coincidence.
You need more of an argument than the word “surely” .
Did you mean “induction” ?
Not under probabilistic reasoning! The hypothesis that text is in English implies a high probability that q’sill be followed by u’s, but not a certainty, there are some exceptions.
So what is non probabilistic uncertainy? Something that doesn’t follow the laws of probability?
But that’s an argument for something general than probability … but the claim that probability theory can found inductionism doesn’t depend on probability being completely general.
(And I said probabilistic reasoning, not Bayes. There are prop!e within Bayes is the one true probability theory and/or a completely general epistemology … But I’m not one of them)
...on your part. The problem is his true scotsmanning. He has these claims that “knowledge is..” , but they are based on defining knowledge, not on making discoveries about the world.
Non explanatory predictive knowledge.
If you mean inductions in an agent occurring without that agent itself making a conjecture ,then yes, I have argued that.
So why are you so sure the claim is true?
and not widely accepted!
Is it simpler? Based on your description, it doesn’t sound simpler. In fact, if you asked me to write a program to conjecture repeating patterns, I would probably end up writing exactly what you would describe as prediction. From what I can tell, this is a distinction without a difference.
I actually don’t need an argument because you are the one claiming a distinction. My claim was an expression of my intuition given your hypothetical, which is meant to query you for said distinction.
Nope.
Here you are making a slight equivocation between different versions of E and different interpretations of probability:
H=The text is English,
E=q’s are always followed by u’s in the text.
That’s different from
H=The text is English,
E′=q’s are followed by u’s in the text with high probability.
In the former case, E is not a logical consequence of H, and so P(E|H) could be less than 1 (though in this sense, probability would not be interpreted as a credence, but as a frequency of letter usage). In the latter case, E’ is a logical consequence of H, so interpreted as credence P(E′|H)=1. In this case, E’ itself is a probabilistic statement (letter frequency) which can be true or false, and it is guaranteed to be true if H is true.
Uncertainty is a subjective feeling, and it still needs to be demonstrated that this feeling can be modeled by probability. I would be careful not to shift the burden of proof. It is the job of the Bayesian to prove we can model all uncertainty this way, not the job of the skeptic to disprove it. As the skeptic, I’ll be careful not to strawman, but if probability really does ground epistemology, then I don’t think it’s a stretch to characterize this as assuming all boolean statements can be taken as inputs to a probability measure, representing our credence. If that’s not the case, I’d be open to an alternative interpretation.
As I mentioned before, bickering over definitions was never Popper’s intention. He was far more interested in explaining the phenomenon of knowledge, and from these pursuits he established his definitions. That’s different from editing one’s definition to exclude arbitrary counterexamples (a No-True-Scotsman). I think we can all agree that the definitions aren’t the point; they are just the facilitators of the communication of phenomena. It’s fine if we disagree on definitions, as long as we agree on the underlying phenomenon.
Given what I just said, perhaps it’s better to rephrase my question: what phenomenon remains unaccounted for without a distinction between inductive reasoning and conjecture? The phenomenon of non-explanatory prediction is classified as a type of conjecture in Popper’s terms.
I’m not sure what you just said.
At the very beginning, I said I lean critical rationalist. I am sure the Popper-Miller theorem is valid given that it is not called a conjecture or a blunder, and anecdotally I have heard mathematicians say it is mathematically obvious (though philosophically confusing), but I am not sure in the sense of understanding it and checking the proof myself. We believe all sorts of things without first-hand corroboration.
Philosophers tend to find the math confusing, and mathematicians tend to find the philosophy confusing. This is not a recipe for acceptance corresponding with corroboration, so that is just an appeal to ignorance.
Yes, obviously. You need to write
a) code to generate conjectures b) code to test the conjectures c) code to reject bad conjectures , and go back a)
Whereas I only need to write b)
Here’s the argument supporting the claim, again:-
“Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up”
Why does that matter?
It sometimes can, since probability sometimes works. Maybe it sometimes d oesn’t, but I sent see how that results in a sweeping deposit of Induction.
Im not defending Bayesianism in that sense, as I said.
Maybe smuggling in definitions without inconvenient bickering was the intention...you are not automatically on the epistemological high ground when you refuse to engage in “semantics”
The ability of agents too simple to form conjectures to nonetheless perform inductive reasoning.
By its authors. But a number of criticisms and counterarguments have been published, eg:-
By the way, an argument can be valid mathematically , but still fail to represent the real world. Conveniently, Vasrani’s argument has that property.
If Popper and Miller have both competencies , others could as well.,
Because you gave an example that didn’t work?
It doesn’t, but it was at least convincing to me that probabilistic reasoning is much more vague than it makes itself out to be.
Sounds good.
Agreed, but choosing to focus on the referent rather than the sense while acknowledging the different senses is the ‘high ground’ as you said, and it is an explicit engaging in semantics. I’m happy to discard or adopt terms if they are shown to be obfuscating or useful respectively.
Perfect, I’m lumping these together because I’m realizing this is the crux and perhaps you can consolidate further. I apologize if I didn’t adequately respond to your other instantiations of these.
For your a)b)c) program, I was only talking about conjectures in that thread, so I would only need to write (a). Is (a) necessarily more complicated than whatever mechanism you have for induction? Also, for me (b) only consists of deductive falsifications, so what you call “induction” would still be part of (a).
For your white box example, it’s not clear to me how initialized expectations are not the same as conjectural dispositions.
For simple agential models which cannot conjecture but still perform inductive reasoning, I’m curious what mechanisms you think are sufficient for conjecture and what mechanisms are necessary for induction? Obviously, for very simple agents, “conjecturing” and “reasoning” aren’t exactly writing down logical statements in English. We’re probably talking about encoding information somehow? Inductive bias, like how ML systems work?
Yeah, and I’ll definitely be looking into those as well. I look forward to it!
Totally agree with the first part.
Definitely, and I hope to be one, but the discourse around it does not inspire confidence.
Why didn’t it work?
Why is that interesting to me? AFAIC ,the debate is about whether induction works. So I’m not interested in general point scoring against Bayes or probability
forming conjectures without any attempt to refute or support them is not knowledge generation.
I’m stipulating that b) is a simple inductor.
No, it’s just doing something in a hard coded way. Not generating an English level description of what to do, interpreting it, and executing it.
Because either you are not updating credence (which I have no objection to), or you can’t distinguish between hypotheses without assuming simplicity as an axiom (which, feel free to do so, but I already argued it doesn’t need to be assumed). But I think this train of thought seems less important than the necessity of induction discussion in the other threads.
It doesn’t need to be. I just found it more compelling.
Totally agree. So I think we may have talked past each other a bit because I was only comparing induction to conjecture, not the full knowledge-generation process. Sure (b) alone is simpler than (a), (b), and (c) collectively, but that’s not what I was arguing against.
Okay, well that’s a bit of a bedrock of disagreement then.
Sure, so what is your sufficient condition for conjecture to be present, and what is your necessary condition for induction to be present?
So have I.:-
(Also, it is completely unclear why “having to assume simplicity” amounts to “not working”. You could argue, as Vasrani does that Bayes without simplicity doesn’t work: I have argued that no real Bayesian ignores simplicity).
Why not? An aircraft without wing s or engine is sim ple, but it can’t fly.
Because you think I was stipulating something else? Because you think there are no simple inductors?
You can tell that a algorithm is making predictions on a black box basis , and you can tell it’s an inductor if it does immediately on boot up.
A conjecture-and-refutation machine has to be complex enough to form high level representations, and make inferences from them.
I think in each of these threads, we’ve started to go in circles, so if it’s any consolation I’m interested in following your future posts, and if I post anything in the future I would be interested to see your critiques.