I’m still confused about what Gear-ness is. I know it is pointing to something, but it isn’t clear whether it is pointing to a single thing, or a combination of things. (I’ve actually been to a CFAR workshop, but I didn’t really get it there either).
Is gear-ness:
a) The extent to which a model allows you to predict a singular outcome given a particular situation? (Ideal situation—fully deterministic like Newtonian physics)
b) The extent to which your model includes each specific step in the causation? (I put my foot on the accelerator → car goes faster. What are the missing steps? Maybe → Engine allows more fuel in → Compressions have greater explosive force → Axels spin faster → Wheels spin faster ->. This could be broken down even further)
c) The extent to which you understand how the model was abstracted out from reality? (ie. You may understand the causation chain and have a formula for describing the situation, but still be unable to produce the proof)
d) The extent to which your understanding of each sub-step has gears-ness?
Honestly, so am I. I think there’s work yet to be done in making the idea of Gears become more Gears-like. I think it has quite a few, but I don’t have a super precise definition that feels to me like it captures the property exactly.
I thought of this when Eliezer sent me a draft of a chapter from a book he was working on. In short (and possibly misrepresenting what he said since it’s been a long time since I’ve read it), he was arguing about how there’s a certain way of seeing what’s true that made him immune to the “sensible” outside-view-like arguments against HPMOR being a worthwhile thing to work on. The arguments he would face, if I remember right, sounded something like this:
“Most fanfics don’t become wildly successful, so yours probably won’t.”
“You haven’t been writing Harry Potter fanfic for long enough to build up a reputation such that others will take your writing seriously.”
“Wait, you haven’t read the canon Rowling books?!? There’s no way you can write good Harry Potter fanfic!” (Yes, seriously. I understand that he maybe still hasn’t read past book 4?)
“Come on, it’s Harry Potter fanfic. There’s no way this matters for x-risk.”
And yet.
(I mean, of course it remains to be seen what will have ultimately mattered, and we can’t compare with much certainty with the counterfactual. But I think it’s totally a reasonable position to think that HPMOR had a meaningful impact on interest in and awareness of x-risk, and I don’t think there’s much room for debate about whether it became a successful piece of fan fiction.)
If I remember right, Eliezer basically said that he understood enough about what engages audiences in a piece of fiction plus how fiction affects people plus how people who are affected by fiction spread the word and get excited by related material that he could see the pathway by which writing HPMOR could be a meaningful endeavor. He didn’t feel terribly affected by people’s attempts to do what he called “reference class tennis” where they would pick a reference class to justify their gut-felt intuition that what he was claiming was sort of beyond his social permissions.
So the query is, What kind of perception of the world and of truth (a) gives this kind of immunity to social pressure when social pressure is out of line and yet (b) will not grant this kind of immunity if culture is more right than we are?
Which reminds me of the kind of debate I was used to seeing in math education research about what it meant to “understand” math, and how it really does feel to me like there’s a really important difference between (a) justifications based “in the math” versus (b) justification based on (even very trustworthy and knowledgeable) other people or institutions.
So, if I trust my intuition on this and assume there really is some kind of cluster here, I notice that the things that feel like more central examples consistently pass the same few tests (the ones I name early in the OP), and the ones that feel like pretty clear non-examples don’t pass those tests very well. I notice that we have something stronger than paying rent from more Gears-like models, and that there’s a capacity to be confused by fiction, and that it seems to restate something about what Eliezer was talking about when the model is “truly a part of you”.
But I don’t really know why. I find that if I start talking about “causal models” or about “how close to physics” the model is or whatever, I end up in philosophical traps that seem to distract from the original generating intuition. E.g., there’s totally a causal model of how the student comes to write the 1 in the addition algorithm, and it seems fraught with philosophical gobbledygook and circular reasoning to specify what about “because the teacher said so” it is that isn’t as “mathematical” as “because you’re summing ones and tens separately”.
I shall endeavor. Eventually the meta-model will be made of Gears too, whatever that turns out to mean. But in the meantime I still think the intuition is super-helpful — and it has the nice property of being self-repairing over time, I think. (I plan on detailing that more in a future post. TL;DR: any “good” process for finding more Gears should be able to be pointed at finding the Gears of Gears in general, and also at itself, so we don’t necessarily have to get this exactly right at the start in order to converge on something right eventually.)
“Seems fraught with philosophical gobbledygook and circular reasoning to specify what about “because the teacher said so” it is that isn’t as “mathematical” as “because you’re summing ones and tens separately”.”
“Because you’re summing ones and tens separately” isn’t really a complete gears level explanation, but a pointer or hint to one. In particular, if you are trying to explain the phenomenon formally, you would begin by defining a “One’s and ten’s representation” of a number n as a tuple (a,b) such that 10a + b = n. We know that at least on such representation exists with a=0 and b=n.
Proof (warning, this is massive, you don’t need to read the whole thing)
You then can define a “Simples one’s and ten’s representation” as such a representation such that 0<=b<=9. We want to show that each number has at least one such representation. It is easy to see that (a, b) = 10a + b = 10a +10 + b − 10 = 10(a+1) + (b-10) = (a+1, b-10). We can repeat this process x times to get (a+x, b-10x). We know that for some x, b-10x will be negative, ie. if x=b, b-10x = −9x. We can decide to look at the last value before it is negative. Let this representation be (m,n). We have defined that n>=0. We also know that n can’t be >=10, else, (m+1, n-10) still has the second element of the tuple >=0. So any number can be written in the tuple form.
Suppose that there are two simple representations of a number (x, y) and (p, q). Then 10x+y = 10p + q. 10(x-p) =y-q. Now, since y and q are between 0 and 9 inclusive, we get that y-q is between 9 and −9, the only factor of 10 in this range is 0. So 10(x-p) =0 meaning x=p and y-q=0, meaning y=q. ie. both members of the tuple are the same.
It is then trivial to prove that (a1, b1) + (a2, b2) = (a1+a2, b1+b2). It is similarly easy to show 0<=b1+b2<=18, so that b1+b2 or b1+b2-10 is between 0 and 9 inclusive. It then directly follow that (a1+a2, b1+b2) or (a1+a2-1, b1+b2-10) is a simple representation (here we haven’t put any restriction on the value of the a’s).
Observations
So a huge amount is actually going on in something so simple. We can make the following observations:
“Because you’re summing ones and tens separately” will seem obvious to many people because they’ve been doing it for so long, but I suspect that the majority of the population would not be able to produce the above proof. In fact, I suspect that the majority of the population would not even realise that it was possible to break down the proof to this level of detail—I believe many of them would see the above sentence as unitary. And even when you tell them that there is an additional level of detail, they probably won’t have any idea what it is supposed to look like.
Part of the reason why it feels more gear like is because it provides you the first step of the proof (defining one’s and ten’s tuples). When someone has a high enough level of maths, they are able to get from the “hint” quite quickly to the full proof. Additionally, even if someone does not have the full proof in their head, they can still see that a certain step will be useful towards producing a proof. The hint of “summing the one’s and tens separately” allows you to quite quickly construct a formal representation of the problem, which is progress even if you are unable to construct a full proof. Discovering that the sum will be between 0 and 18, let’s you know that if you carry, you will only ever have to carry the one. This limits the problem to a more specific case. Any person attempting to solve this will probably have examples in the past where limiting the case in such a way made the proof either easier or possible, so whatever heurestic pattern matching which occurs within their brain will suggest that this is progress (though it may of course turn out later that the ability to restrict the situation does not actually make the proof any easier)
Another reason why it may feel more gear like is that it is possible to construct sentence of a similar form and use them as hints for other proofs. So, “Because you’re summing ones and tens separately” is linguistically close to “Because you’re summing tens and hundreds separately”, although I don’t want to suggest that people only perform a linguistic comparison. If someone has developed an intuitive understand of these phenomenon, this will also play a role.
I believe that part of the reason why it is so hard to define what is or what is not “gears-like” is because this isn’t based on any particular statement or model just by itself, but in terms of how this interacts with what a person already knows and can derive in order to produce statements. Further, it isn’t just about producing one perfect gears explanation, but the extent to which a person can produce certain segments of the proof (ie. a formal statement of the problem or restricting the problem to the sub-case as above) or the extent to which it allows the production of various generalisation (ie. we can generalise to (tens & hundreds, hundreds and thousands… ) or to (ones & tens & hundreds) or to binary or to abstract algebra). Further, what counts as a useful generalisation is not objective, but relative to the other maths someone knows or the situations that someone knows in which they can apply this maths. For example, imaginary numbers may not seem like a useful generalisation until a person knows the fundamental theorem of algebra or how it can be used to model phases in physics.
I won’t claim that I’ve completely or even almost completely mapped out the space of gears-ness, but I believe that this takes you pretty far towards an understanding of what it might be.
I think this is part of it but the main metaphor is more like “your model has no hand-wavy-ness. There are clear reasons that the parts connect to each other, that you can understand as clearly as you can understand how gears connect to each other.”
I’m still confused about what Gear-ness is. I know it is pointing to something, but it isn’t clear whether it is pointing to a single thing, or a combination of things. (I’ve actually been to a CFAR workshop, but I didn’t really get it there either).
Is gear-ness:
a) The extent to which a model allows you to predict a singular outcome given a particular situation? (Ideal situation—fully deterministic like Newtonian physics)
b) The extent to which your model includes each specific step in the causation? (I put my foot on the accelerator → car goes faster. What are the missing steps? Maybe → Engine allows more fuel in → Compressions have greater explosive force → Axels spin faster → Wheels spin faster ->. This could be broken down even further)
c) The extent to which you understand how the model was abstracted out from reality? (ie. You may understand the causation chain and have a formula for describing the situation, but still be unable to produce the proof)
d) The extent to which your understanding of each sub-step has gears-ness?
Honestly, so am I. I think there’s work yet to be done in making the idea of Gears become more Gears-like. I think it has quite a few, but I don’t have a super precise definition that feels to me like it captures the property exactly.
I thought of this when Eliezer sent me a draft of a chapter from a book he was working on. In short (and possibly misrepresenting what he said since it’s been a long time since I’ve read it), he was arguing about how there’s a certain way of seeing what’s true that made him immune to the “sensible” outside-view-like arguments against HPMOR being a worthwhile thing to work on. The arguments he would face, if I remember right, sounded something like this:
“Most fanfics don’t become wildly successful, so yours probably won’t.”
“You haven’t been writing Harry Potter fanfic for long enough to build up a reputation such that others will take your writing seriously.”
“Wait, you haven’t read the canon Rowling books?!? There’s no way you can write good Harry Potter fanfic!” (Yes, seriously. I understand that he maybe still hasn’t read past book 4?)
“Come on, it’s Harry Potter fanfic. There’s no way this matters for x-risk.”
And yet.
(I mean, of course it remains to be seen what will have ultimately mattered, and we can’t compare with much certainty with the counterfactual. But I think it’s totally a reasonable position to think that HPMOR had a meaningful impact on interest in and awareness of x-risk, and I don’t think there’s much room for debate about whether it became a successful piece of fan fiction.)
If I remember right, Eliezer basically said that he understood enough about what engages audiences in a piece of fiction plus how fiction affects people plus how people who are affected by fiction spread the word and get excited by related material that he could see the pathway by which writing HPMOR could be a meaningful endeavor. He didn’t feel terribly affected by people’s attempts to do what he called “reference class tennis” where they would pick a reference class to justify their gut-felt intuition that what he was claiming was sort of beyond his social permissions.
So the query is, What kind of perception of the world and of truth (a) gives this kind of immunity to social pressure when social pressure is out of line and yet (b) will not grant this kind of immunity if culture is more right than we are?
Which reminds me of the kind of debate I was used to seeing in math education research about what it meant to “understand” math, and how it really does feel to me like there’s a really important difference between (a) justifications based “in the math” versus (b) justification based on (even very trustworthy and knowledgeable) other people or institutions.
So, if I trust my intuition on this and assume there really is some kind of cluster here, I notice that the things that feel like more central examples consistently pass the same few tests (the ones I name early in the OP), and the ones that feel like pretty clear non-examples don’t pass those tests very well. I notice that we have something stronger than paying rent from more Gears-like models, and that there’s a capacity to be confused by fiction, and that it seems to restate something about what Eliezer was talking about when the model is “truly a part of you”.
But I don’t really know why. I find that if I start talking about “causal models” or about “how close to physics” the model is or whatever, I end up in philosophical traps that seem to distract from the original generating intuition. E.g., there’s totally a causal model of how the student comes to write the 1 in the addition algorithm, and it seems fraught with philosophical gobbledygook and circular reasoning to specify what about “because the teacher said so” it is that isn’t as “mathematical” as “because you’re summing ones and tens separately”.
I shall endeavor. Eventually the meta-model will be made of Gears too, whatever that turns out to mean. But in the meantime I still think the intuition is super-helpful — and it has the nice property of being self-repairing over time, I think. (I plan on detailing that more in a future post. TL;DR: any “good” process for finding more Gears should be able to be pointed at finding the Gears of Gears in general, and also at itself, so we don’t necessarily have to get this exactly right at the start in order to converge on something right eventually.)
“Seems fraught with philosophical gobbledygook and circular reasoning to specify what about “because the teacher said so” it is that isn’t as “mathematical” as “because you’re summing ones and tens separately”.”
“Because you’re summing ones and tens separately” isn’t really a complete gears level explanation, but a pointer or hint to one. In particular, if you are trying to explain the phenomenon formally, you would begin by defining a “One’s and ten’s representation” of a number n as a tuple (a,b) such that 10a + b = n. We know that at least on such representation exists with a=0 and b=n.
Proof (warning, this is massive, you don’t need to read the whole thing)
You then can define a “Simples one’s and ten’s representation” as such a representation such that 0<=b<=9. We want to show that each number has at least one such representation. It is easy to see that (a, b) = 10a + b = 10a +10 + b − 10 = 10(a+1) + (b-10) = (a+1, b-10). We can repeat this process x times to get (a+x, b-10x). We know that for some x, b-10x will be negative, ie. if x=b, b-10x = −9x. We can decide to look at the last value before it is negative. Let this representation be (m,n). We have defined that n>=0. We also know that n can’t be >=10, else, (m+1, n-10) still has the second element of the tuple >=0. So any number can be written in the tuple form.
Suppose that there are two simple representations of a number (x, y) and (p, q). Then 10x+y = 10p + q. 10(x-p) =y-q. Now, since y and q are between 0 and 9 inclusive, we get that y-q is between 9 and −9, the only factor of 10 in this range is 0. So 10(x-p) =0 meaning x=p and y-q=0, meaning y=q. ie. both members of the tuple are the same.
It is then trivial to prove that (a1, b1) + (a2, b2) = (a1+a2, b1+b2). It is similarly easy to show 0<=b1+b2<=18, so that b1+b2 or b1+b2-10 is between 0 and 9 inclusive. It then directly follow that (a1+a2, b1+b2) or (a1+a2-1, b1+b2-10) is a simple representation (here we haven’t put any restriction on the value of the a’s).
Observations
So a huge amount is actually going on in something so simple. We can make the following observations:
“Because you’re summing ones and tens separately” will seem obvious to many people because they’ve been doing it for so long, but I suspect that the majority of the population would not be able to produce the above proof. In fact, I suspect that the majority of the population would not even realise that it was possible to break down the proof to this level of detail—I believe many of them would see the above sentence as unitary. And even when you tell them that there is an additional level of detail, they probably won’t have any idea what it is supposed to look like.
Part of the reason why it feels more gear like is because it provides you the first step of the proof (defining one’s and ten’s tuples). When someone has a high enough level of maths, they are able to get from the “hint” quite quickly to the full proof. Additionally, even if someone does not have the full proof in their head, they can still see that a certain step will be useful towards producing a proof. The hint of “summing the one’s and tens separately” allows you to quite quickly construct a formal representation of the problem, which is progress even if you are unable to construct a full proof. Discovering that the sum will be between 0 and 18, let’s you know that if you carry, you will only ever have to carry the one. This limits the problem to a more specific case. Any person attempting to solve this will probably have examples in the past where limiting the case in such a way made the proof either easier or possible, so whatever heurestic pattern matching which occurs within their brain will suggest that this is progress (though it may of course turn out later that the ability to restrict the situation does not actually make the proof any easier)
Another reason why it may feel more gear like is that it is possible to construct sentence of a similar form and use them as hints for other proofs. So, “Because you’re summing ones and tens separately” is linguistically close to “Because you’re summing tens and hundreds separately”, although I don’t want to suggest that people only perform a linguistic comparison. If someone has developed an intuitive understand of these phenomenon, this will also play a role.
I believe that part of the reason why it is so hard to define what is or what is not “gears-like” is because this isn’t based on any particular statement or model just by itself, but in terms of how this interacts with what a person already knows and can derive in order to produce statements. Further, it isn’t just about producing one perfect gears explanation, but the extent to which a person can produce certain segments of the proof (ie. a formal statement of the problem or restricting the problem to the sub-case as above) or the extent to which it allows the production of various generalisation (ie. we can generalise to (tens & hundreds, hundreds and thousands… ) or to (ones & tens & hundreds) or to binary or to abstract algebra). Further, what counts as a useful generalisation is not objective, but relative to the other maths someone knows or the situations that someone knows in which they can apply this maths. For example, imaginary numbers may not seem like a useful generalisation until a person knows the fundamental theorem of algebra or how it can be used to model phases in physics.
I won’t claim that I’ve completely or even almost completely mapped out the space of gears-ness, but I believe that this takes you pretty far towards an understanding of what it might be.
I think this is part of it but the main metaphor is more like “your model has no hand-wavy-ness. There are clear reasons that the parts connect to each other, that you can understand as clearly as you can understand how gears connect to each other.”