My best insight is that you can think. Because most of the failure comes from not thinking. … I was making these kinds of excuses for years… and I made basically no progress before I just tried.”
I resonate with this. Cognitive psychology and sequences taught me to be extremely mistrustful of my own thoughts, and while I do think much of that was a necessary first step, I also think it’s very easy to get stuck in that mindset because you’ve disowned the only tools that can save you. Non-cognition is very tempting if your primary motivation is to not be wrong—especially when that mindset is socially incentivised.
Introspection is the most reliable source of knowledge we have, but it isn’t publishable/sharable/legible, so very few people use it to its full potential. (point made a week ago)
The popular evo-psych idea “self-deception for the purpose of other-deception” is largely a myth. We were never privy to our intentions in the first place, so there’s nothing for self-deception to explain. Our “conscious mind” is us looking at ourselves from an outside perspective, and we may only infer our true intentions via memory traces we’re lucky enough to catch a glimpse of. (two weeks ago)
I stress introspection so much because it’s ~futile to make novel object-level progress without a deep familiarity/understanding of the tools you use for the task.
Do it every day, at least a little bit.
This particular sentence I’m sorta out of phase with feeling-wise, however. I’m patiently trying to cultivate intrinsic motivation by… a large list of complicated subjective tricks most of which are variations on “minimise total motivational force spent on civil war between parts of myself.” It’s like politics, or antibiotic resistance—if I overreach in an attempt to eliminate a faction I’m still too weak to permanently defeat, it’s likely to backfire.
The Light is more powerfwl, though Darkness is quicker, easier, more seductive.
This particular sentence I’m sorta out of phase with feeling-wise, however. I’m patiently trying to cultivate intrinsic motivation by… a large list of complicated subjective tricks most of which are variations on “minimise total motivational force spent on civil war between parts of myself.” It’s like politics, or antibiotic resistance—if I overreach in an attempt to eliminate a faction I’m still too weak to permanently defeat, it’s likely to backfire.
I completely agree with this approach. It is just that starting is the hardest part and if do something every day at least a little bit, you will make it a lot easier to start on command. That is one of the advantages. Normally thinking about alignment for 1 minute is not causing an internal war I expect. I think having this goal is good, and there is no conflict necessarily with what you are talking about. I think it is best if both are combined.
The popular evo-psych idea “self-deception for the purpose of other-deception” is largely a myth.
It is very real in my experience. In hindsight, I have caught myself many times. Maybe I mean something different than you. Motivated early stopping would fall into this for me, when you are sort of not aware/suppressing to become aware of that you are doing this. Which I think is the default. I am very sure I have observed my mind suppressing further thought once it early stopped with some ridiculous justification.
I stress introspection so much because it’s ~futile to make novel object-level progress without a deep familiarity/understanding of the tools you use for the task.
I think this is wrong. I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did. Though I agree that I could probably have done better with introspection.
I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did.
More specifically, I mean progress wrt some long-term goal like AI alignment, altruism, factory farming, etc. Here, I think most ways of thinking about the problem are wildly off-target bc motivations get distorted by social incentives. Whereas goals in narrow games like “win at chess” or “solve a math problem” are less prone to this, so introspection is much less important.
Well, I am talking about creating games, not playing them if that was unclear. I think that is significantly harder than making games. It took over a thousand hours of practice to get good. I think AI alignment is a lot harder, but I think the same pattern applies to some extent. For example, asking the question “What will this project look like if it goes really well is a good question.” Why well when John asked this question to a bunch of people he got good results. I have not thought about why you get good results, but asking this question. But I am pretty sure I could understand it better, and that is likely to be useful, compared to not understanding. But clearly, you can get benefits even when you don’t understand.
Most of the time when you are applying a technique you will just be applying the technique. You will normally not retrieve all of the knowledge of why this technique works before using it. And it works fine. The knowledge about why the technique is mostly useful for refining the technique is my guess. However, applying the refined technique does not require retrieving the knowledge. In fact, you might often forget the knowledge but not the refined technique, i.e. the procedural knowledge.
I resonate with this. Cognitive psychology and sequences taught me to be extremely mistrustful of my own thoughts, and while I do think much of that was a necessary first step, I also think it’s very easy to get stuck in that mindset because you’ve disowned the only tools that can save you. Non-cognition is very tempting if your primary motivation is to not be wrong—especially when that mindset is socially incentivised.
Introspection is the most reliable source of knowledge we have, but it isn’t publishable/sharable/legible, so very few people use it to its full potential. (point made a week ago)
The popular evo-psych idea “self-deception for the purpose of other-deception” is largely a myth. We were never privy to our intentions in the first place, so there’s nothing for self-deception to explain. Our “conscious mind” is us looking at ourselves from an outside perspective, and we may only infer our true intentions via memory traces we’re lucky enough to catch a glimpse of. (two weeks ago)
I stress introspection so much because it’s ~futile to make novel object-level progress without a deep familiarity/understanding of the tools you use for the task.
This particular sentence I’m sorta out of phase with feeling-wise, however. I’m patiently trying to cultivate intrinsic motivation by… a large list of complicated subjective tricks most of which are variations on “minimise total motivational force spent on civil war between parts of myself.” It’s like politics, or antibiotic resistance—if I overreach in an attempt to eliminate a faction I’m still too weak to permanently defeat, it’s likely to backfire.
The Light is more powerfwl, though Darkness is quicker, easier, more seductive.
I completely agree with this approach. It is just that starting is the hardest part and if do something every day at least a little bit, you will make it a lot easier to start on command. That is one of the advantages. Normally thinking about alignment for 1 minute is not causing an internal war I expect. I think having this goal is good, and there is no conflict necessarily with what you are talking about. I think it is best if both are combined.
It is very real in my experience. In hindsight, I have caught myself many times. Maybe I mean something different than you. Motivated early stopping would fall into this for me, when you are sort of not aware/suppressing to become aware of that you are doing this. Which I think is the default. I am very sure I have observed my mind suppressing further thought once it early stopped with some ridiculous justification.
I think this is wrong. I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did. Though I agree that I could probably have done better with introspection.
More specifically, I mean progress wrt some long-term goal like AI alignment, altruism, factory farming, etc. Here, I think most ways of thinking about the problem are wildly off-target bc motivations get distorted by social incentives. Whereas goals in narrow games like “win at chess” or “solve a math problem” are less prone to this, so introspection is much less important.
Well, I am talking about creating games, not playing them if that was unclear. I think that is significantly harder than making games. It took over a thousand hours of practice to get good. I think AI alignment is a lot harder, but I think the same pattern applies to some extent. For example, asking the question “What will this project look like if it goes really well is a good question.” Why well when John asked this question to a bunch of people he got good results. I have not thought about why you get good results, but asking this question. But I am pretty sure I could understand it better, and that is likely to be useful, compared to not understanding. But clearly, you can get benefits even when you don’t understand.
Most of the time when you are applying a technique you will just be applying the technique. You will normally not retrieve all of the knowledge of why this technique works before using it. And it works fine. The knowledge about why the technique is mostly useful for refining the technique is my guess. However, applying the refined technique does not require retrieving the knowledge. In fact, you might often forget the knowledge but not the refined technique, i.e. the procedural knowledge.