I definitely agree that most people most of the time are using an algorithm like babble-with-associations, with feedback refining the associations over time. That said, I do think it is possible to do better, and at least some people (especially STEM people working on hard problems) learn to use other methods at least sometimes. In the rationality-movement-I’d-like-to-see, systematizing and training such more-efficient-thinking-algorithms is a central piece of the curriculum, and a major goal of the movement.
I’ve been thinking about explicit models of more-efficient-thinking-algorithms for a while now—Mazes and Crayon was an early piece along those lines, and of course a lot of the gears pieces are tied in. I have at least one thinking-algorithm which is more efficient on novel problems than vanilla babble-with-associations and which I think is trainable. I’ve been working towards a write-up (Everyday Lessons from High-Dimensional Optimization was partly intended as background for that write-up).
That write-up isn’t ready yet, but here are some sparknotes:
We use gears-level models because black-box problem solving in high dimensions is inefficient. The idea of “gears” is to decompose the system into coupled low-dimensional subsystems.
My current best guess is that equations/constraints are a universal representation for gears. E.g. in a causal model (specifically structural equations), the equations/constraints are the structural equations themselves. Likewise in physics/chemistry/engineering/etc: the constraints are the system’s governing equations. When we want to problem-solve on the system, those governing equations act as constraints on our problem.
I expect that humans’ native representation of gears-level models amounts to something equivalent to constraints as well.
This immediately suggests a whole class of problem-solving strategies, in particular:
constraint relaxation to generate heuristics (e.g. “could I solve this problem ignoring this constraint?”)
dual methods (e.g. thinking about tautness/slackness of constraints—which constraints are “easy”, and which are limiting our ability to solve the problem)
There’s also the question of how to figure out the equations/constraints in the first place. Presumably we could look for constraints-on-the-constraints, but then we go up a whole ladder of meta-levels… except we actually don’t. Here again, we can leverage duality: constraints-on-constraints are partial solutions to the original problem. So as we go “up” the meta-ladder, we just switch back-and-forth between two problems. One specific example of this is simultaneously looking for a proof and a counterexample in math. A more visual example is in this comment.
Concretely, this process looks like trying out some solution-approaches while looking for any common barrier they run into, then thinking about methods to specifically circumvent that barrier (ignoring any other constraints), then looking for any common barrier those methods run into, etc...
The key feature of this class of thinking-algorithms is that they can get around the inefficiency issues in high-dimensional problems, as long as the problem-space decomposes—which real-world problem spaces usually do, even in cases where you might not expect it. This is basically just taking some standard tricks from AI (constraint relaxation, dual methods), and applying them directly to human thinking.
Note that this can still involve babbly thinking for the lower-level steps; the whole thing can still be implemented on top of babble-with-associations. The key is that we only want to babble at low-dimensional subproblems, while using a more systematic approach for the full high-dimensional problem.
This all sounds true (and I meant it to be sort of implied by the post, although I didn’t delve into every given possible “improved algorithm”, and perhaps could have picked a better example.)
What seemed to me was the gears/model-based-thinking still seems implemented on babble, not just for the lower level steps, but for the higher level systematic strategy. (I do think this involves first building some middle-order thought processes on top of the babble, and then building the high level strategy out of those pieces)
i.e. when I use gears-based-systematic-planning, the way the pieces of the plan come together still feel like they’re connected via the same underlying associative babbling. It’s just that I’d have a lot of tight associations between collections of strategies, like:
Notice I’m dealing with a complex problem
Complex problem associates into “use the appropriate high level strategy for this problem” (which might involve first checking possible strategies, or might involve leaping directly to the correct strategy)
Once I have a gears-oriented strategy, it’ll usually have a step one, then a step two, etc (maybe looping around recursively, or with branching paths) and each step is closely associated with the previous step.
Does it feel differently to you?
When you do the various techniques you describe above, what is the qualia and low-level execution of it feel like?
I do think it’s all ultimately implemented on top of something babbly, yeah. The babbly part seems like the machine code of the brain—ultimately everything has to be implemented in that.
I think what I mean by “gearsy reasoning” is something different than how you’re using the phrase. It sounds like you’re using it as a synonym for systematic or system-2 reasoning, whereas I see gears as more specifically about decomposing systems into their parts. Gearsy reasoning doesn’t need to look very systematic, and systematic reasoning doesn’t need to be gearsy—e.g. simply breaking things into steps is not gearsy reasoning in itself. So the specific “tight associations” you list do not sound like the things I associate with gearsy thinking specifically.
As an example, let’s say I’m playing a complicated board game and figuring out how to get maximum value out of my resources. The thought process would be something like:
Ok, main things I want are X, Y, Z → what resources do I need for all that?
(add it up)
I have excess A and B but not enough C → can I get more C?
I have like half a dozen ways of getting more C, it’s basically interchangeable with B at a rate of 2-to-1 → do I have enough B for that?
...
So that does look like associative babbling; the “associations” it’s following are mainly the relationships between objects given by the game actions, plus the general habit of checking what’s needed (i.e. the constraints) and what’s available.
I guess one insight from this: when engaged in gears-thinking, it feels like the associations are more a feature of the territory than of my brain. It’s not about taste, it’s about following the structure of reality (or at least that’s how it feels).
I think what I mean by “gearsy reasoning” is something different than how you’re using the phrase. It sounds like you’re using it as a synonym for systematic or system-2 reasoning, whereas I see gears as more specifically about decomposing systems into their parts.
Yeah. My reply was somewhat general and would work for non-gearsy strategies as well. I do get that gearsiness and systematicness are different axes and strategies can employ them independently. I was referring offhandedly to “systematic gearsiness” because it’s what you had just mentioned and I just meant to convey that the babble-process worked for it.
i.e, I think your list that begins “Okay, the main things I want are X, Y and Z...” follows naturally from my list that ends “Once I have a gears-oriented strategy, it’ll usually have a step one...”
I guess one insight from this: when engaged in gears-thinking, it feels like the associations are more a feature of the territory than of my brain. It’s not about taste, it’s about following the structure of reality.
The way I’d parse it is that I have some internalized taste that “when figuring out a novel, complex problem, it’s important to look for associations that are entangled with reality”. And then as I start exploring possible strategies to use, or facts that might be relevant, “does this taste gearsy?” and “does this taste ‘entangled with reality’” are useful things to be able to check. (Having an aesthetic taste oriented around gearsy-entangledness lets you quickly search or rule out directions of thought at the sub-second level, which might then turn into deliberate, conscious thought)
Alternately: I’m developing a distaste for “babbling that isn’t trying to be methodical” when working on certain types of problems, which helps remind me to move in a more methodical direction (which is often but not always gearsy)
[edit: I think you can employ gearsy strategies without taste, I just think taste is a useful thing to acquire
I definitely agree that most people most of the time are using an algorithm like babble-with-associations, with feedback refining the associations over time. That said, I do think it is possible to do better, and at least some people (especially STEM people working on hard problems) learn to use other methods at least sometimes. In the rationality-movement-I’d-like-to-see, systematizing and training such more-efficient-thinking-algorithms is a central piece of the curriculum, and a major goal of the movement.
I’ve been thinking about explicit models of more-efficient-thinking-algorithms for a while now—Mazes and Crayon was an early piece along those lines, and of course a lot of the gears pieces are tied in. I have at least one thinking-algorithm which is more efficient on novel problems than vanilla babble-with-associations and which I think is trainable. I’ve been working towards a write-up (Everyday Lessons from High-Dimensional Optimization was partly intended as background for that write-up).
That write-up isn’t ready yet, but here are some sparknotes:
We use gears-level models because black-box problem solving in high dimensions is inefficient. The idea of “gears” is to decompose the system into coupled low-dimensional subsystems.
My current best guess is that equations/constraints are a universal representation for gears. E.g. in a causal model (specifically structural equations), the equations/constraints are the structural equations themselves. Likewise in physics/chemistry/engineering/etc: the constraints are the system’s governing equations. When we want to problem-solve on the system, those governing equations act as constraints on our problem.
I expect that humans’ native representation of gears-level models amounts to something equivalent to constraints as well.
This immediately suggests a whole class of problem-solving strategies, in particular:
constraint relaxation to generate heuristics (e.g. “could I solve this problem ignoring this constraint?”)
dual methods (e.g. thinking about tautness/slackness of constraints—which constraints are “easy”, and which are limiting our ability to solve the problem)
There’s also the question of how to figure out the equations/constraints in the first place. Presumably we could look for constraints-on-the-constraints, but then we go up a whole ladder of meta-levels… except we actually don’t. Here again, we can leverage duality: constraints-on-constraints are partial solutions to the original problem. So as we go “up” the meta-ladder, we just switch back-and-forth between two problems. One specific example of this is simultaneously looking for a proof and a counterexample in math. A more visual example is in this comment.
Concretely, this process looks like trying out some solution-approaches while looking for any common barrier they run into, then thinking about methods to specifically circumvent that barrier (ignoring any other constraints), then looking for any common barrier those methods run into, etc...
The key feature of this class of thinking-algorithms is that they can get around the inefficiency issues in high-dimensional problems, as long as the problem-space decomposes—which real-world problem spaces usually do, even in cases where you might not expect it. This is basically just taking some standard tricks from AI (constraint relaxation, dual methods), and applying them directly to human thinking.
Note that this can still involve babbly thinking for the lower-level steps; the whole thing can still be implemented on top of babble-with-associations. The key is that we only want to babble at low-dimensional subproblems, while using a more systematic approach for the full high-dimensional problem.
This all sounds true (and I meant it to be sort of implied by the post, although I didn’t delve into every given possible “improved algorithm”, and perhaps could have picked a better example.)
What seemed to me was the gears/model-based-thinking still seems implemented on babble, not just for the lower level steps, but for the higher level systematic strategy. (I do think this involves first building some middle-order thought processes on top of the babble, and then building the high level strategy out of those pieces)
i.e. when I use gears-based-systematic-planning, the way the pieces of the plan come together still feel like they’re connected via the same underlying associative babbling. It’s just that I’d have a lot of tight associations between collections of strategies, like:
Notice I’m dealing with a complex problem
Complex problem associates into “use the appropriate high level strategy for this problem” (which might involve first checking possible strategies, or might involve leaping directly to the correct strategy)
Once I have a gears-oriented strategy, it’ll usually have a step one, then a step two, etc (maybe looping around recursively, or with branching paths) and each step is closely associated with the previous step.
Does it feel differently to you?
When you do the various techniques you describe above, what is the qualia and low-level execution of it feel like?
I do think it’s all ultimately implemented on top of something babbly, yeah. The babbly part seems like the machine code of the brain—ultimately everything has to be implemented in that.
I think what I mean by “gearsy reasoning” is something different than how you’re using the phrase. It sounds like you’re using it as a synonym for systematic or system-2 reasoning, whereas I see gears as more specifically about decomposing systems into their parts. Gearsy reasoning doesn’t need to look very systematic, and systematic reasoning doesn’t need to be gearsy—e.g. simply breaking things into steps is not gearsy reasoning in itself. So the specific “tight associations” you list do not sound like the things I associate with gearsy thinking specifically.
As an example, let’s say I’m playing a complicated board game and figuring out how to get maximum value out of my resources. The thought process would be something like:
Ok, main things I want are X, Y, Z → what resources do I need for all that?
(add it up)
I have excess A and B but not enough C → can I get more C?
I have like half a dozen ways of getting more C, it’s basically interchangeable with B at a rate of 2-to-1 → do I have enough B for that?
...
So that does look like associative babbling; the “associations” it’s following are mainly the relationships between objects given by the game actions, plus the general habit of checking what’s needed (i.e. the constraints) and what’s available.
I guess one insight from this: when engaged in gears-thinking, it feels like the associations are more a feature of the territory than of my brain. It’s not about taste, it’s about following the structure of reality (or at least that’s how it feels).
Yeah. My reply was somewhat general and would work for non-gearsy strategies as well. I do get that gearsiness and systematicness are different axes and strategies can employ them independently. I was referring offhandedly to “systematic gearsiness” because it’s what you had just mentioned and I just meant to convey that the babble-process worked for it.
i.e, I think your list that begins “Okay, the main things I want are X, Y and Z...” follows naturally from my list that ends “Once I have a gears-oriented strategy, it’ll usually have a step one...”
The way I’d parse it is that I have some internalized taste that “when figuring out a novel, complex problem, it’s important to look for associations that are entangled with reality”. And then as I start exploring possible strategies to use, or facts that might be relevant, “does this taste gearsy?” and “does this taste ‘entangled with reality’” are useful things to be able to check. (Having an aesthetic taste oriented around gearsy-entangledness lets you quickly search or rule out directions of thought at the sub-second level, which might then turn into deliberate, conscious thought)
Alternately: I’m developing a distaste for “babbling that isn’t trying to be methodical” when working on certain types of problems, which helps remind me to move in a more methodical direction (which is often but not always gearsy)
[edit: I think you can employ gearsy strategies without taste, I just think taste is a useful thing to acquire