teenager | mathematics enthusiast | MIT class of 2026 | vaguely Grey Triber | personal website: https://duck-master.github.io
duck_master
I think that if you imagine the deity Murphy trying to foil your plans whatever you do, that gives you a pretty decent approximation to true infraBayesianism.
Here’s an ELI5: The evil superintelligent deity Murphy, before you were ever conceived, picked the worst possible world that you could live in (meaning the world where your performance is worst), and you have to use fancy math tricks to deal with that.
More specifically: if two points are in a convex set, then the entire line segment connecting them must also be in the set.
Me too. I currently only have a very superficial understanding of infraBayesianism (all of which revolves around the metaphysical, yet metaphorical, deity Murphy).
“Mixture of infra-distributions” as in convex set, or something else? If it’s something else then I’m not sure how to think about it properly.
...and it’s closed.
Don’t worry, it was kind of a natural stopping point anyways, as the discussion was winding down.
@Diffractor: I think I got a MIRIxDiscord invite in a way somehow related to this event. Check your PMs for details. (I’m just commenting here to get attention because I think this might be mildly important.)
I agree with you (meaning G Gorden Worley III) that Wikipedia is reliable, and I too treat it as reliable. (It’s so well-known as a reliable source that even Google uses it!) I also agree that an army of bots and humans undo any defacing that may occur, and that Wikipedia having to depend on other sources helps keep it unbiased. I also agree with the OP that Wikipedia’s status as not-super-reliable among the Powers that Be does help somewhat.
So I think that the actual secret of Wikipedia’s success is a combination of the two: Mild illegibility prevents rampant defacement, citations do the rest. If Wikipedia was both viewed as Legibly Completely Accurate and also didn’t cite anything, then it would be defaced to hell and back and rendered meaningless; but even if everyone somehow decided one day that Wikipedia was ultra-accurate and also that they had a supreme moral imperative to edit it, I still think that Wikipedia would still turn out okay as a reliable source if it made the un-cited content very obvious, e.g. if each [citation needed] tag was put in size 128 Comic Sans and accompanied by an earrape siren* and even if there was just a bot that put those tags after literally everything without a citation**. (If Wikipedia is illegible, of course it’s going to be fine.)
*I think trolls might work around this by citing completely unrelated things, but this problem sounds like it could be taken care of by humans or by relatively simple NLP.
**This contravenes current Wikipedia policy, but in the worst-case scenario of Ultra-Legible Wikipedia, I think it would quickly get repealed.
I think this applies to every wiki ever, and also to this very site. There are probably a lot of others that I’m missing but this is a start.
Bumping this.
Bumping it again.
Au contraire, I think that “mutual information between the object and the environment” is basically the right definition of “knowledge”, at least for knowledge about the world (as it correctly predicts that all four attempted “counterexamples” are in fact forms of knowledge), but that the knowledge of an object also depends on the level of abstraction of the object which you’re considering.
For example, for your rock example: A rock, as a quantum object, is continually acquiring mutual information with the affairs of humans by the imprinting of subatomic information onto the surface of rock by photons bouncing off the Earth. This means that, if I was to examine the rock-as-a-quantum-object for a really long time, I would know the affairs of humans (due to the subatomic imprinting of this information on the surface of the rock), and not only that, but also the complete workings of quantum gravity, the exact formation of the rock, the exact proportions of each chemical that went into producing the rock, the crystal structure of the rock, and the exact sequence of (micro-)chips/scratches that went into making this rock into its current shape. I feel perfectly fine counting all this as the knowledge of the rock-as-a-quantum-object, because this information about the world is stored in the rock.
(Whereas, if I were only allowed to examine the rock-as-a-macroscopic-object, I would still know roughly what chemicals it was made of and how they came to be, and the largest fractures of the rock, but I wouldn’t know about the affairs of humans; hence, such is the knowledge held by the rock-as-a-macroscopic-object. This makes sense because the rock-as-a-macroscopic-object is an abstraction of the rock-as-a-quantum-object, and abstractions always throw away information except that which is “useful at a distance”.)
For more abstract kinds of knowledge, my intuition defaults to question-answering/epistemic-probability/bet-type definitions, at least for sufficiently agent-y things. For example, I know that 1+1=2. If you were to ask me, “What is 1+1?”, I would respond “2″. If you were to ask me to bet on what 1+1 was, in such a way that the bet would be instantly decided by Omega, the omniscient alien, I would bet with very high probability (maybe 40:1odds in favor, if I had to come up with concrete numbers?) that it would be 2 (not 1, because of Cromwell’s law, and also because maybe my brain’s mental arithmetic functions are having a bad day). However, I do not know whether the Riemann Hypothesis is true, false, or independent of ZFC. If you asked me, “Is the Riemann Hypothesis true, false, or independent of ZFC?”, I would answer, “I don’t know” instead of choosing one of the three possibilities, because I don’t know. If you asked me to bet on whether the Riemann Hypothesis was true, false, or independent of ZFC, with the bet to be instantly decided by Omega, I might bet 70% true, 20% false, and 10% independent (totally made-up semi-plausible figures that no bearing on the heart of the argument; I haven’t really tested my probabilistic calibration), but I wouldn’t put >95% implied probability on anything because I’m not that confident in any one possibility. Thusly, for abstract kinds of knowledge, I think I would say that an agent (or a sufficiently agent-y thing) knows an abstract fact X if it tells you about this fact when prompted with a suitably phrased question, and/or if it places/would place a bet in favor of fact X with very high implied probability if prompted to bet about it.
(One problem with this definition is that, intuitively, when I woke up today, I had no idea what 384384*20201 was; the integers here are also completely arbitrary. However, after I typed it into a calculator and got 7764941184, I now know that 384384*20201 = 7764941184. I think this is also known as the problem of logical omniscience; Scott Aaronson once wrote a pretty nice essay about this topic and others from the perspective of computational complexity.)
I have basically no intuition whatsoever on what it means for a rock* to know that the Riemann Hypothesis is true, false, or independent of ZFC. My extremely stupid and unprincipled guess is that, unless a rock is physically inscribed with a proof of the true answer, it doesn’t know, and that otherwise it does.
*I’m using a rock here as a generic example of a clearly-non-agentic thing. Obviously, if a rock was an agent, it’d be a very special rock, at least in the part of the multiverse that I inhabit. Feel free to replace “rock” with other words for non-agents.
Since this is a literally a question about soliciting predictions, it should have one of those embedded-interactive-predictions-with-histograms gadgets* to make predicting easier. Also, it might be worth it to have two prediction gadgets, since this is basically a prediction: one gadget to predict what Recognized AI Safety Experts (tm) predict about how much damage unsafe AIs will do, and one gadget to predict about how much damage unsafe AIs will actually do (to mitigate weird second-order effects having to do with predicting a prediction).
*I’m not sure what they’re supposed to be called.
I think knowledge as a whole cannot be absent, but knowledge of a particular fact can definitely be absent (if there’s no relationship between the thing-of-discourse and the fact).
Thanks for the reply. I take it that not only are you interested in the idea of knowledge, but that you are particularly interested in the idea of actionable knowledge.
Upon further reflection, I realize that all of the examples and partial definitions I gave in my earlier comment can in fact be summarized in a single, simple definition: a thing X has knowledge of a fact Y iff it contains some (sufficiently simple) representation of Y. (For example, a rock knows about the affairs of humans because it has a representation of those affairs in the form of Fisher information, which is enough simplicity for facts-about-the-world.) Using this definition, it becomes much easier to define actionable knowledge: a thing X has actionable knowledge of a fact Y iff it contains some sufficiently simple representation of Y, and this representation is so simple that an agent with access to this information could (with sufficiently minimal difficulty) make actions that are based on fact Y. (For example, I have actionable knowledge that 1 + 1 = 2, because my internal representation of this fact is so simple that I can literally type up its statement in a comment.) It also becomes clearer that actionable knowledge and knowledge are not the same (since, for example, the knowledge about the world that a computer that records cryptographic hashes of everything it observes could not be acted upon without breaking the hashes, which is presumably infeasible).
So as for the human psychology/robot vacuum example: If your robot vacuum’s internal representation of human psychology is complex (such as in the form of video recordings of humans only), then it’s not actionable knowledge and your robot vacuum can’t act on it; if it’s sufficiently simple, such as a low-complexity-yet-high-fidelity executable simulation of a human psyche, your robot vacuum can. My intuition also suggests in this case that your robot vacuum’s knowledge of human psychology is actionable iff it has a succinct representation of the natural abstraction of “human psychology” (I think this might be generalizable; i.e. knowledge is actionable iff it’s succinct when described in terms of natural abstractions), and that finding out whether your robot vacuum’s knowledge is sufficiently simple is essentially a matter of interpretability. As for the betting thing, the simple unified definition that I gave in the last paragraph should apply as well.
- 26 Jun 2021 1:39 UTC; 1 point) 's comment on Problems facing a correspondence theory of knowledge by (
I also agree that, for the purpose of previewing the content, this post is poorly titled (maybe it should be titled something like “Having bad names makes you open the black box of the name”, except more concise?), although, for me, I didn’t as much stick to a particular wrong interpretation as just view the entire title as unclear.
Maybe flow?
In fact, organized resources like Wikipedia, LW sequences, SEP, etc. are basically amortized scholarship. (This is particularly true for Wikipedia; its entire point is that we find vaguely-related content from around—or beyond—the web and then paraphrase it into a mildly-coherent article. Source: am wikipedia editor.)
Google doc where we posted our confusions/thoughts earlier: https://docs.google.com/document/d/1lKG_y_Voe02OkRGG9yaxtMuGM_dQBUKjj9DXTA8rMxE/edit
My ongoing confusions/thoughts:
What if the super intelligent deity is less than maximally evil or maximally good? (E.g. the deity picking the median-performance world)
What about the dutch-bookability of infraBayesians? (the classical dutch-book arguments seem to suggest pretty strongly that non-classical-Bayesians can be arbitrarily exploited for resources)
Is there a meaningful metaphysical interpretation of infraBayesianism that does not involve Murphy? (similarly to how Bayesianism can be metaphysically viewed as “there’s a real, static world out there, but I’m probabilistically unsure about it”)