Discord: LemonUniverse (.lemonuniverse). Reddit: u/Smack-works. About my situation: here.
I wrote some bad posts before 2024 because I was very uncertain how the events may develop.
I do philosophical/conceptual research, have no mathematical or programming skills. But I do know a bunch of mathematical and computer science concepts.
What I’m trying to say is
We don’t have to view “God exists” as an analytic statement.[1]
If something is not an analytic statement, it still could be proven true by a priori reasoning (according to Kantian philosophy). Non-analytic statements which are a priori true are called synthetic a priori. They don’t require physical evidence to be justified.
I don’t see why the argument fails if we consider “God exists” as a synthetic a priori statement.
IMO your response doesn’t address this.
“God is the most perfect thing” could be an analytic truth, while “an existing thing is more perfect than a nonexistent thing” could be a synthetic a priori truth. Then “God is a thing that exists” would follow as another syntehtic a priori truth.
Why does the argument fail and why can’t a very similar one be made? Erich quotes:
However, if the statement is synthetic, the ontological argument does not work, as the existence of God is not contained within the definition of God (and, as such, evidence for God would need to be found).
If the existence of God is a synthetic a priori judgement, then the argument itself could be all the evidence we need. Synthetic a priori judgements don’t need some physical evidence.
Could God be a synthetic a priori statement? I tried googling, but not much discussion and it seems out of my league to understand why Kant rejected it.
If merely replacing the premise “god is X” with “if god exists then it’s X” is enough to break the ontological argument, it’s clear the argument was a swindle.
Since when does it work this way? There are many paradoxes which could be defeated by replacing something with something (the two envelopes problem might be the most notorious), but the challenge is to explain why the original thing doesn’t work.
Here’s my proposed solution: just use probability everywhere.
Sounds too good to be true. What are the pitfalls?
Math, Rationality, Funny quotes from SSC/ACX—feel like too small, unoriginal topics. Meta—skipped this section. Political philosophy—didn’t get it. AI—how does Platonism connect to persuasion?
Without context it doesn’t really tell much, because even some simple things can be drawn as complex diagrams. But perhaps the only way to learn context is to do serious study...
question: to what extent is biology evidence for the modularity assumption?
I’m also interested in a more general question: how do analogies provide evidence? how can we get evidence about computation X from a potentially unrelated computation Y? how do vague abstractions over types of computations work (for example, is there a semi-rigorous definition of “numbers”? what would it be good for?)?
on the other hand, metabolic pathways seem ridiculously complicated, and it often seems like everything is mangled up with everything else in a complicated way.
Is there a popular resource which can give a layman a feel of this complexity?
Everyone who watched every Rob video ever should also read the lost Arbital Seuqnece, though it’s older.
A famous folk criticism of school goes like this:
“School knowledge is hard to learn because it’s not useful for real-life goals.”
I think it’s wrong. IMO the real problem is
“School knowledge is hard to learn because it’s not useful for any goal at all, not even a fictional one.”
At least it’s presented this way. Let me explain what I mean.
For knowledge (= a set of facts) to be useful and meaningful, three conditions need to be met:
The facts are useful for solving a simple/natural problem. Or, at the very least, the facts should be about a simple/natural topic.
There are “super facts”, useful for deducing many other facts.
You can notice what super facts you don’t know and deduce their potential value.
1st condition guarantees that your knowledge is about something. 2nd condition guarantees that your knowledge is systematically useful, not just accidentally useful due to special circumstances. 3rd condition guarantees that your knowledge is more than a bunch of random heuristics, that you can do minimal meta-cognition about your knowledge.
The conditions apply to knowledge about anything, from boxing to abstract math.
For LOTR, the simple problem is “kill Sauron”. It contextualizes most facts about the story:
Knowing properties of the One Ring is broadly useful. Because it can be a weapon against Sauron, a shortcut for killing him, or a cursed burden for the heroes.
Knowing Frodo’s physical/mental/moral abilities is broadly useful. Because he can help to handle the Ring and he has to survive a grueling journey.
Knowing Gandalf’s abilities is broadly useful. Because he can fight, protect, provide information and have political influence.
Knowing geography is broadly useful. Because it determines the shortest and safest path to Mordor + determines the best/worst places for fighting the Sauron’s army.
If you don’t know something from this list, you can notice that you don’t know it and realize the potential usefulness of what you don’t know. For example, you can go “wait, what kind of being is Gandalf? what exactly are his powers? are there other guys like him? can they help the heroes? is Sauron like Gandalf? why is Gandalf weaker? or is he weaker?...” All of that is pretty useful for knowing why and how you can (not) just kill Sauron.
For chess, the simple problem is “checkmate the enemy king”. It contextualizes most facts about the game:
Knowing tactics is broadly useful. Because they pop up everywhere (opening, middlegame, endgame) and allow fast checkmates/wins of material. Also, some strategic ideas only work because of tactics.
Knowing endgames is broadly useful. Because most games end in endgames + because the prospect of a particular endgame can shape the middlegame and even the opening play.
Knowing opening theory is broadly useful. Because every game starts in an opening + openings can shape middlegame and endgame play.
Knowing stuff about pawn structures is broadly useful. Because ~all openings/middlegames/endgames depend on them.
If you are bad at something from the list, you can reflect on how it hinders you (your ability to checkmate the enemy king) and how to play around your weaknesses.
For AI alignment, the problem is “build a superintelligence that doesn’t kill everyone”.
Even such galaxy-brained effort like Principia Mathematica was about a simple problem—“minimize the amount of concepts and avoid paradoxes”.
School knowledge is not built around simple/natural problems. It’s built around a tautologically defined problem of solving tests. “You have to know X to solve the test because we put X on the test.” E.g. “you have to know when this guy died because we gonna ask you when this guy died”.
There’s no meaningful goal (real or fictional) to apply the knowledge to, so no way to notice that you don’t know something and realize the potential value of knowing it.
English Learning
In English learning classes (English is a foreign language for me) they were making us memorize different tenses & moods before we had any chance to understand how important each grammatical structure was.
That was making the subject seem unnecessarily scary, because in reality one could speak/read English perfectly without using many of those structures. I still remember grappling with some structure which I ended up… just successfully avoiding. Didn’t stop me from writing philosophy in English and getting some $$$ for it.
I’m not saying “obscure” structures shouldn’t be learned, just that it’s wrong to learn facts without learning to contextualize them.
7th Grade Physics
Here’s my review of a classic russian textbook by Alexandr V. Peryshkin. My version was somewhere from the 00s.
1) The intro tells what Physics studies (changes of bodies and substances) and what types of physical phenomena there are (mechanical, electrical, magnetic, thermal, acoustic and optical).
The book seems to only talk about mechanical phenomena. It’s never stated explicitly though and “mechanical” is left undefined. Assuming the actual topic is mechanical phenomena, the difference between mechanical and thermal/acoustic is not explained: the book talks about particles, gas and water, so why not temperature (particles’ movement) and sound (matter vibrations)?
2) The book entirely excludes chemical reactions (e.g. fire) from physics. Doesn’t justify it.
3) The book jumps between basic facts about movement (inertia, acceleration, potential energy, work) and stuff like deformation/pressure/friction. This could be justified as studying “the most widespread & noticeable & simple ways physical bodies/substances affect each other”, but the book doesn’t try to justify anything.
4) A couple of times the book gets unexpectedly specific (Hooke’s law, Archimedes’ principle, law of the lever). Jumps in specificity are not justified. No attempt to define what subset of mechanical phenomena we’re studying.
5) The book talks about weight, solid pressure, liquid pressure, force composition—but doesn’t bring it all together. There’s little explanation of how pressure works differently for solids and liquids.
Conclusion: most of the book is not based on any defined topic (I’m not talking about a rigorous definition, just any definition at all). There’s no big picture, no frame to put the puzzle pieces in. The information doesn’t add up to a whole.
Conclusions from my view of learning:
A) To learn something from school, your only options are to become a fanatic (be re-learning the subjects on your own through sheer passion) or violate your brain (by cramming useless, disjointed facts into it).
As a consequence, people decide to avoid science (because learning endless lists of useless facts is a pretty unpleasant/impossible undertaking—and the school makes it seem like it’s necessary for learning science) or fall for pseudoscience (because it’s the first science-like worldview they encounter which has at least fictional usefulness).
B) Effective learning can be highly standardized.
C) Learning doesn’t have to be super rigorous, formalist.
D) Practice (doing homework) is not always necessary for learning.
I want a fully general definition of “understanding” to understand how minds work. To understand what makes a mind unified.
Disambiguating world models is a good motivation to focus on messiness/clarity.
I think of interesting problems as being those whose answers (you think will) most help you disambiguate your global world model.
Unexpected and interesting definition idea. Will look forward to future posts.
I actually think of “questions” and “problems” as being fairly neat concepts.
Makes sense. IIRC many abstraction researchers (like Sam Eisenstat or John Wentworth) do the same.
One objection though: your definition doesn’t define what is an interesting question/problem.
Regarding ontology...I think of flywheels as being small, toy domains which give unusually clear, unambiguous answers, which can aid in the construction of theories applicable to messier domains. “High resolution” parts of reality, if you will.
So domains can be bigger/smaller (in terms of state space), harder/easier (in terms of computational complexity) or messier/clearer (in terms of measurement error and other problems with result interpretation). The last distinction seems less fundamental.[1] Do you really need to focus on it?
Separate question: would you say that e.g. Gettier cases are a toy subdomain of philosophy, allowing to test different definitions of knowledge?
(I’m trying to probe your conceptualization of flywheels in different ways to maybe help you write down some thoughts or inspire new ideas. If it doesn’t help feel free to say so or just ignore this message.)
“Messiness” seems to pop up only when you try to interpret domain A through domain B, but you’re confused about how to do it (e.g. you don’t know statistics) or lack capabilities to do it (e.g. you don’t have precise enough instruments).
I guess I wanted to ask… in what direction do you want to take the concept?
1.5.1 Modernity has seen an explosion of our scientific and mathematical knowledge. The core mechanism behind this can be described as follows: virtuous feedback loops between conceptual progress and empirical/calculational flywheels.
1.5.2 What is a empirical/calculational flywheel? By this I mean a set of well-defined procedures and technologies which enable one to pose precise questions and problems about some aspect of physical or mathematical reality.
You introduce the concept of a flywheel. It’s connected to “feedback loops”, “technology”, “questions”, “problems”… but all of those auxiliary concepts are pretty ugly. I mean from a mathematical/ontological perspective. Because they are pretty complicated and contingent.
So one way you could develop the concept is to “purify” it, simplify it as much as possible, make it as ontologically fundamental as possible. Define what a “flywheel” is independently from feedback loops / technology / questions, or in a way which generalizes all those auxiliary concepts.
Combining two flywheels into one...hmm, maybe. I would be a bit more general and say that deep discoveries compress a lot of your worldview, which could include uniting two small well-defined domains(or showing a particular small domain well predicts a bigger domain)
I mean one direction you could take the concept of flywheels in is to try to define “(deep) knowledge” in terms of flywheels. Or something like that.
But the two ideas above are not the only valid ways to develop the idea of flywheels. Maybe you want to do something entirely different with them. Or maybe they are just not an important enough concept to develop like this.
Oops! Sorry, I ended up rediscovering America.
Mesa- is a Greek prefix which means the opposite of meta-. To “go meta” is to go one level up; to “go mesa” is to go one level down (nobody has ever actually used this expression, sorry). So a mesa-optimizer is an optimizer one level down from you. (c.) Scott Alexander
But I first encountered this reading in a Rob Miles’ video. However, Rob’s slides actually show that it’s not valid Greek and the prefix literal meaning is different. (Though there’s also some non-AI paper which conceptualizes meta/mesa as opposites.) But I forgot/didn’t notice. And the original post does point out the prefix’ literal meaning too.
When I googled around before making the shortform, I found this stackexchange post where people argue what should be the opposite of meta and this reddit post.
In mesa-optimizers, “mesa-” is not actually a real prefix meaning the opposite of “meta-”, illustrating the deceptiveness of those things.
You suspected something was off, but the prefix promised way too many capabilities and conveniences for you to question it.
edit: oops, see this
“Flywheel” seems like the most interesting concept here.
Does it mean something like “a well-defined way to measure something”, “representation/subset of the problem with a smaller state-space”? For example, integers is a flywheel because it’s a well-defined way to measure some (but not all) properties of objects. Thermometer is a flywheel because it’s a well-defined way to measure some (but not all) properties of the body.
Any deep discovery combines multiple very different flywheels into one. (Maybe. I’m just spitballing.) Like when Descartes discovered that two number lines define a plane (in such a way that many shapes have simple equations). Or like when the concept of a group was introduced, combining symmetries and something number-like. Or like when Newton realized that the same thing can explain both orbiting motion and falling/throwing.
Is this aligned with your thinking?
Analyzing abstractions, semantics and human language is a sorta famous research direction in alignment. (The researchers are John Wentworth & David Lorell & Gretta Duleba, Thane Rutheins, Jan Kulveit, Sam Eisenstat.)
So I wrote a draft where I tried to
classify ALL interpretable types of human abstractions,
explain where the usefulness of different abstraction types comes from,
explain how polysemantic words work (I analyzed “run”, “set” and “put”).
The idea is to give a fully general definition of what it means for a set of abstractions/computations to be “interpretable”.
Here’s the draft:
I’d like to get feedback, because the topic is complicated and I don’t want to fumble it. If you want to give some, feel free to message me in LW DMs/on Discord or comment here/in the doc.
TIL what a “zero day” is! Was always confused because of the Y2K problem—I thought zero days have something to do with calendars. (That didn’t make sense but I somehow didn’t consciously thought about the confusion.)
Many of my current research interests are about “unification” or “molecules” (by “molecules” I mean something like semantic molecules in semantic molecularism).
“What unifies computations about X into ‘understanding’ of X?”
“What unifies computations into an ontology?”
“What unifies computations into a mind?”
“What unifies specific intellectual skills into general intellectual skills?”
“What unifies computations about a word into the ‘meaning’ of the word?”
Below are some thought experiments and observations related to those questions.
Understanding
Imagine you don’t know anything about calculus.
Someone gives you a black box which can solve calculus problems. Do you understand calculus know? No.
Someone connects the black box directly to your brain, so you can use it in your mind. Do you understand calculus know? No, because it’s still a black box relative to the rest of your mind.
Someone lets you inspect the code of the black box and you can understand individual instructions. Do you understand calculus know? Not necessarily. Understanding details doesn’t mean you have a model of the whole.
Conclusion. There should be some definition of “understanding”. And some thing which unifies the knowledge of details into a single gestalt. The above thought experiment is very similar to the Chinese room thought experiment (especially the systems reply discussion), though my focus is very different.
Outrospection
Imagine taking a computation (within a bigger computation), giving it the ability to memorize its computations and the ability to “look around itself”. What would it see?
For example, imagine giving the calculator (within your mind) memory and the ability to see the rest of your mind. What would it see? It could notice some simple arithmetic patterns in your brain activity.[1]
The same question applies to consciousness (because it’s smaller than the entire brain) and to humans (because they’re smaller than the entire universe, both in terms of size and coarse-grained perception).
If you’re interested, here’s an entire post draft about the question “what does a set of computations see around itself?”—https://docs.google.com/document/d/1uvdQzkcilqV2T-EbN2AaukpSPIc-E-g9rtxgVuhih68/edit?tab=t.0
Mind Center
Even if something is significantly dumber than you, it still can plot against you and sabotage you. This applies to both people and brain sub-computations.
Therefore, all coherent minds have to have a center (to the extent to which those minds are not coherent by accident) - a minimal computation which can check the alignment of all the other computations. “What determines a mind’s effects?” by TsviBT should be related.
Note. The idea that self-alignment requires corrigibility is not new (see e.g. this comment and this post).
Conditions of Understanding
I feel like we can find necessary and sufficient conditions for “understanding”.
Here’s one idea. For a set of facts to create “understanding”, three conditions need to be met:
The facts should be useful for solving a simple/natural problem. Or, at the very least, the facts should be about a simple/natural topic.
There should be general facts, useful for deducing many other facts.
You should be able notice what general facts you don’t know and deduce their potential value[2]
The conditions apply to knowledge about anything, from boxing to abstract math. Here’s a shortform which goes into examples.
Question. The above seems like necessary, but not sufficient conditions. Because you cam come up with meaningless sets of facts which fit those conditions. What’s missing? Maybe something related to composition, the ability to combine facts.
Reasoning Transparency
Consider two chess heuristics:
The player with material advantage will often win. Even a small advantage (one or two pawns) will often be enough to win.
The player with material advantage will often increase their material advantage. Even a small advantage (one or two pawns) will often increase.
Both are heuristics—informal, unproven statements. However, the 2nd heuristic is also an explanation of the 1st heuristic: if we assume that (2) is true and (1) is at least a little bit true, then it follows that (1) is fully true.
Moreover, the 2nd heuristic divides the 1st into milestones. Making it easier to evaluate whether the 1st heuristic is true for a specific position. Probably any safe reasoning should be divisible into milestones.
Conclusion. We can study how heuristic reasoning connects to logic; how disparate heuristics connect into a single understanding; how unproven reasoning can be transparent. This line of reasoning should connect to heurstic explanations in some way.
Learning
Imagine a game where you play as a frog. You can control the length and the angle of your jumps. The goal is to jump on a blue spot somewhere on a map. The less jumps you use, the better. There’s no way to redo a level.
You complete level after level. Then in one level the physics of frog jumping suddenly changes. If you rely on old instincts, you constantly overshoot or undershoot. How to reach the blue dot now?
Consider two goals:
Reach the blue dot.
Decrease the distance to the blue dot.
The 2nd goal allows to split the 1st goal into steps, notice which step doesn’t work (increases your distance instead of decreasing) and relearn your strategy for that particular step.
Conclusion. Note how the learning trick here is very similar to the transparency trick in the previous section. Does something like this generalize to all learning?
Learning Patterns
Imagine a game where you play as a mouse. You move (up/down, right/left and diagonally) on a 2D plane. The goal is to lead the mouse to the cheese. The faster, the better. There’s no way to redo a level.
Consider three goals:
Moving closer to the cheese.
Moving up/down.
Moving right/left.
The 2nd and 3rd goals allow to decompose the 1st goal into parts (i.e. into a horizontal and a vertical components), learn a pattern like “moving diagonally is faster than moving all the way up/down and all the way right/left” (Euclidean distance) and relearn that pattern if it’s no longer true (e.g. if the distance metric in a level changed to Taxicab).
Conclusion. If this idea generalizes to all learning, then it’s a necessary component of general intelligence.
This is like a reverse of introspection—outrospection. Though in some sense all introspection is outrospection.
1st condition guarantees that your knowledge is about something. 2nd condition guarantees that your knowledge is systematically useful, not just accidentally useful due to special circumstances. 3rd condition guarantees that your knowledge is more than a bunch of random heuristics, that you can do minimal reflection (= meta-cognition) about your knowledge.