MS in mathematics, and philosophy amateur, I generally align myself with critical rationalist views (apologies to my Bayesian friends), and socially liberal values.
JugglingJay
Karma: 0
Why didn’t it work?
Because either you are not updating credence (which I have no objection to), or you can’t distinguish between hypotheses without assuming simplicity as an axiom (which, feel free to do so, but I already argued it doesn’t need to be assumed). But I think this train of thought seems less important than the necessity of induction discussion in the other threads.
Why is that interesting to me?
It doesn’t need to be. I just found it more compelling.
forming conjectures without any attempt to refute or support them is not knowledge generation.
Totally agree. So I think we may have talked past each other a bit because I was only comparing induction to conjecture, not the full knowledge-generation process. Sure (b) alone is simpler than (a), (b), and (c) collectively, but that’s not what I was arguing against.
I’m stipulating that b) is a simple inductor.
Okay, well that’s a bit of a bedrock of disagreement then.
No, it’s just doing something in a hard coded way. Not generating an English level description of what to do, interpreting it, and executing it.
Sure, so what is your sufficient condition for conjecture to be present, and what is your necessary condition for induction to be present?
Why does that matter?
Because you gave an example that didn’t work?
...but I cant* see how that results in a sweeping deposit of Induction.
It doesn’t, but it was at least convincing to me that probabilistic reasoning is much more vague than it makes itself out to be.
I’m not defending Bayesianism in that sense, as I said.
Sounds good.
...you are not automatically on the epistemological high ground when you refuse to engage in “semantics”
Agreed, but choosing to focus on the referent rather than the sense while acknowledging the different senses is the ‘high ground’ as you said, and it is an explicit engaging in semantics. I’m happy to discard or adopt terms if they are shown to be obfuscating or useful respectively.
“Whereas I only need to write b)...
″...Here’s the argument supporting the claim, again:-
‘Note that because its a white box, you can can show there is is no time T, in its execution, where the conjecture that the patterns will repeat is formed, as opposed to a previous time where it hasn’t ….It expects repeating patterns from boot up’...”
″...The ability of agents too simple to form conjectures to nonetheless perform inductive reasoning...”
Perfect, I’m lumping these together because I’m realizing this is the crux and perhaps you can consolidate further. I apologize if I didn’t adequately respond to your other instantiations of these.
For your a)b)c) program, I was only talking about conjectures in that thread, so I would only need to write (a). Is (a) necessarily more complicated than whatever mechanism you have for induction? Also, for me (b) only consists of deductive falsifications, so what you call “induction” would still be part of (a).
For your white box example, it’s not clear to me how initialized expectations are not the same as conjectural dispositions.
For simple agential models which cannot conjecture but still perform inductive reasoning, I’m curious what mechanisms you think are sufficient for conjecture and what mechanisms are necessary for induction? Obviously, for very simple agents, “conjecturing” and “reasoning” aren’t exactly writing down logical statements in English. We’re probably talking about encoding information somehow? Inductive bias, like how ML systems work?
But a number of criticisms and counterarguments have been published
Yeah, and I’ll definitely be looking into those as well. I look forward to it!
By the way, an argument can be valid mathematically , but still fail to represent the real world. Conveniently, Vasrani’s argument has that property.
Totally agree with the first part.
If Popper and Miller have both competencies , others could as well.
Definitely, and I hope to be one, but the discourse around it does not inspire confidence.
That is simpler than writing something that conjectures repeating patterns...
Is it simpler? Based on your description, it doesn’t sound simpler. In fact, if you asked me to write a program to conjecture repeating patterns, I would probably end up writing exactly what you would describe as prediction. From what I can tell, this is a distinction without a difference.
You need more of an argument than the word “surely” .
I actually don’t need an argument because you are the one claiming a distinction. My claim was an expression of my intuition given your hypothetical, which is meant to query you for said distinction.
Did you mean “induction” ?
Nope.
Not under probabilistic reasoning! The hypothesis that text is in English implies a high probability that q’s will* be followed by u’s, but not a certainty, there are some exceptions.
Here you are making a slight equivocation between different versions of E and different interpretations of probability:
The text is English,
q’s are always followed by u’s in the text.
That’s different from
The text is English,
q’s are followed by u’s in the text with high probability.
In the former case, E is not a logical consequence of H, and so could be less than 1 (though in this sense, probability would not be interpreted as a credence, but as a frequency of letter usage). In the latter case, E’ is a logical consequence of H, so interpreted as credence . In this case, E’ itself is a probabilistic statement (letter frequency) which can be true or false, and it is guaranteed to be true if H is true.
So what is non probabilistic uncertainy? Something that doesn’t follow the laws of probability?
Uncertainty is a subjective feeling, and it still needs to be demonstrated that this feeling can be modeled by probability. I would be careful not to shift the burden of proof. It is the job of the Bayesian to prove we can model all uncertainty this way, not the job of the skeptic to disprove it. As the skeptic, I’ll be careful not to strawman, but if probability really does ground epistemology, then I don’t think it’s a stretch to characterize this as assuming all boolean statements can be taken as inputs to a probability measure, representing our credence. If that’s not the case, I’d be open to an alternative interpretation.
...on your part. The problem is his true scotsmanning. He has these claims that “knowledge is..” , but they are based on defining knowledge, not on making discoveries about the world.
As I mentioned before, bickering over definitions was never Popper’s intention. He was far more interested in explaining the phenomenon of knowledge, and from these pursuits he established his definitions. That’s different from editing one’s definition to exclude arbitrary counterexamples (a No-True-Scotsman). I think we can all agree that the definitions aren’t the point; they are just the facilitators of the communication of phenomena. It’s fine if we disagree on definitions, as long as we agree on the underlying phenomenon.
Non explanatory predictive knowledge.
Given what I just said, perhaps it’s better to rephrase my question: what phenomenon remains unaccounted for without a distinction between inductive reasoning and conjecture? The phenomenon of non-explanatory prediction is classified as a type of conjecture in Popper’s terms.
If you mean inductions in an agent occurring without that against itself making making a conjecture ,then yes, I have argued that.
I’m not sure what you just said.
So why are you so sure the claim is true?
At the very beginning, I said I lean critical rationalist. I am sure the Popper-Miller theorem is valid given that it is not called a conjecture or a blunder, and anecdotally I have heard mathematicians say it is mathematically obvious (though philosophically confusing), but I am not sure in the sense of understanding it and checking the proof myself. We believe all sorts of things without first-hand corroboration.
and not widely accepted!
Philosophers tend to find the math confusing, and mathematicians tend to find the philosophy confusing. This is not a recipe for acceptance corresponding with corroboration, so that is just an appeal to ignorance.
But induction can be performed by organisms and software too simple to form conjectures.
I’m actually not as well informed on how reasoning operates in other organisms, but if you are allowing for primitive structures that enable some kind of proto-inductive reasoning, then I have no idea why you wouldn’t also allow for primitive structures that enable proto-conjecture. If there’s a distinction between the two, then surely conjecture would operate on even more primitive mechanics than inductive reasoning. Automated systems are mostly doing optimization, which is sort of in a totally different camp, but I’d allow for the possibility that something like Attention in LLMs is simulating some kind of intuition. Still, that’s a total guess on my part.
Maybe that’s a True Scotsman argument...This* also sounds like a true Scotsman.
Neither were No-True-Scotsman’s, since I defined what Popper meant by “reasoning” and “knowledge” in my very first post, so these exclusions were not merely arbitrary.
You don’t need to assume it because you can argue for it methodologically*.
I’m certainly interested in diving into this more, since I find the math fascinating. Nonetheless, in my 3rd post when I said:
...unless you want to reason to it...
I was giving you the option to accept it axiomatically vs. deriving it, and perhaps it’s on me that I interpreted your following responses as taking the former position. However, if you derive it, you must derive it utilizing probabilistic reasoning (as it sounds like you do), which runs into the issue I mentioned; namely , because that’s what evidence means. Evidence is a logical consequence of the hypothesis, so , or if you want to view them as sets of possible worlds, . This immediately implies . This result ruins the desired Bayesian updating for induction, which was the whole reason why you mentioned simplicity, which was the reason I assumed it was your axiom.
However, a better critique would be the very notion of probability as uncertainty. (This argument is my own) Probability is merely normalized measure defined on some -algebra (a definition of measurable sets). It can be used to model measurement errors, frequencies of outcomes, perhaps the number of indiscernible symmetries of a system, and some uncertainties if it is well understood that the model is accurate. However, to model all uncertainty as purely probabilistic as if that is the one true philosophic interpretation of probability I find overly presumptuous. What makes Bayesians think the set of all subjective uncertainties can be modeled via some -algebra? I’ll grant we don’t need to know the density function (that’s the point of Bayesian statistics), but we should at least know the -algebra so we can know to what to apply probability.
Unless it’s conjecture.
Again, see the definition of reasoning I mentioned above.
We can’t account for the production of all kinds of knowledge without induction, because it produces one of the kinds.
Which kind of knowledge does it produce that is not produced deductively?
But I believe it is a thing, because it can be demonstrated directly , and because there* are not any sound and valid arguments* against it that I have seen.
Is there any demonstrations of induction that are not merely conjecture? I would say the argument above is valid, and also from the link I shared. If you disagree, point to the logical error.
What is the best argument against it, and why not quote* it directly?
Frankly, because I don’t know it. I don’t even know the technical statement, let alone its proof. What I presented here is my understanding, and the link I provided is a better articulation of the extent of my understanding. The Popper-Miller Theorem is notoriously difficult to understand and frequently misconstrued, so I chose to point to one that better represented my understanding. Feel free to look it up, but since I have enough expertise in logic and mathematics to be confident I could understand it, as well as an interest to eventually understand it, I’m not going to be particularly moved by any LW criticisms of the theorem until I do so.
Why is that a problem? There is still a form of probabilistic inductive reasoning that works.
I’m just articulating Popper’s views to the best of my ability, and he did not believe in probabilistic induction. He explains what we call ‘inductive reasoning’ as mere conjecture (perhaps calibrating for some psychological biases), but that’s not actually reasoning (at least to Popper). The reason assuming simplicity as a fundamental criterion is a ‘problem’ as you said, is because you don’t need to. As I mentioned, Popper can account for the desirability of simplicity without assuming it as an axiom.
That isn’t telling me induction doesn’t work.
That wasn’t the argument against induction. It was an accounting for simplicity without assuming it.
Induction is useful...In any case “induction isn’t needed” is a different claim to “induction doesn’t work”.
Again, just through a Popperian lens: prediction is useful, even without explanation. However, prediction on its own is not knowledge. For Popper, induction was just never a thing to begin with. I think my point in saying “induction was never needed in the first place” was to emphasize that we can still account for knowledge production without induction. I agree it’s different from saying “induction doesn’t work,” but if the logic of induction is not warranted (perhaps by showing it doesn’t mathematically work), then induction isn’t a thing.
It’s a mistake to suppose that just because you have two different” schools”, they can’t agree on a single point.
Totally agree, but without induction, guess-and-check is perhaps the most primitive way of describing Popperianism. To be a little triggering, induction would change it to semi-prescient guess-and-check. Regardless, I agree this is a bit of an oversimplification.
Sure, but that adds the additional assumption about simplicity, and it concedes that evidence doesn’t weigh more in one generalization’s favor over another. Bayesianism requires this extra axiom, which ironically makes it less simple (unless you want to reason to it from Bayes’ Theorem, but that ignores the fact that should be ). In contrast, simplicity is desired on a Popperian account because it makes hypotheses easier to test; clear simple predictions are usually easier to falsify.
I think Popper’s point is that induction was never needed in the first place. Knowledge grows through the process I mentioned, and we don’t need to assume any inductive trick that necessarily (even probabilistically) gets us to general laws. We can accept that knowledge is fundamentally trial-and-error or guess-and-check, and the phenomenon of knowledge-production loses nothing.
He has multiple objections to induction. One is that it is not deductively valid, which in fact is addressed by reframing it in probabilistic terms.
I’m not so sure it is addressed by reframing in probabilistic terms. At least for now, I’m convinced of a version of Popper’s argument against probabilistic reasoning (admittedly, probably an over-simplified version), and I plan to familiarize myself with the more formal argument in the near future.
Solomonoff induction, if you could make it work, would address that issue of knowledge creation as well, since Solomonoff Inductors generate hypotheses mechanically.
That would be fine for hypothesis-generation, and perhaps that is how our minds do it when we do so rationally, but I believe it is Popper’s view that this isn’t the same as knowledge-generation. Perhaps it generates good hypotheses, but the scrutinizing step is still necessary for knowledge. Popper is effectively saying that a Monte Carlo random search through hypotheses could still generate good hypotheses, and we would still be able to extract knowledge from them if the scrutiny step is applied. Knowledge need not require an effective algorithm (though I’m sure any strategy is better than none).
If something is only in a book, and not in anyone’s head, is it known? Is it not merely potential knowledge?
I think that is one of the fundamental differences in perspective. However, I’ll be a little vague here because Popper didn’t care as much about the one true definition of a thing, but rather its function and its explanation. So the question shouldn’t be “Is it known?” but rather “What is going on when we say it is known?”
I don’t even think you need the universe/multiverse to be deterministic. Even if there are fundamentally random aspects of the universe, we would still need to consolidate those features that are similar and those whose unpredictability can be modeled by statistics, like you said. As for prediction markets, I think their efficacy may be overblown, but I would need to look into it more.
This is a bit of an old post, but I felt I might be able to add to the discussion. Keep in mind this is my own informal take on a rigorous philosophical topic, and I am by no means a professional. My bias leans towards critical rationalism (Popperianism), but I’ll try to be fair.
I think you are correct in identifying induction as the fundamental tension between the two epistemologies. Bayesian epistemology (as distinct from Bayes’ Theorem) utilizes Solomonoff induction, whereas Popper is highly critical of inductive and probabilistic reasoning. For Popper, induction isn’t reasoning at all, if we view reasoning as a method by which we generate knowledge. I’m sometimes tempted to call it “pseudo-inductive reasoning,” which is just the fancy way of saying “guessing.”
To understand Popper’s views, it’s important to understand that knowledge for Popper has nothing to do with belief, credence, or confidence; those are all subjective states of one’s mind. Perhaps we can psychologically manipulate these mental states (perhaps upon realizing the Bayesian calculus), but those mental states don’t constitute knowledge.
If we view the scientific endeavor in three pieces, it might become more clear: 1) We propose ideas, hypotheses, and conjectures 2) We scrutinize these hypotheses and eliminate the falsified ones, and 3) We adopt those hypotheses which have been highly scrutinized as our new scientific theories, acting as if they are true until they are usurped. For Popper, only (2) is the knowledge-generation process, while Bayesians greatly concern themselves with (1) and (3) (they don’t neglect (2), but they don’t see it as identically the essence of knowledge-generation).
Popper gave a simple example of how knowledge can be contained inside books, and so is not intrinsically tied to our beliefs or actions. Rather, (in my understanding) knowledge is the record of the progress of falsification, and the hypotheses at the top of the current leaderboard. How we generated the hypotheses and what we do with our theories once we have them are problems for psychology and rationality, not epistemology.
I think in each of these threads, we’ve started to go in circles, so if it’s any consolation I’m interested in following your future posts, and if I post anything in the future I would be interested to see your critiques.