This particular sentence I’m sorta out of phase with feeling-wise, however. I’m patiently trying to cultivate intrinsic motivation by… a large list of complicated subjective tricks most of which are variations on “minimise total motivational force spent on civil war between parts of myself.” It’s like politics, or antibiotic resistance—if I overreach in an attempt to eliminate a faction I’m still too weak to permanently defeat, it’s likely to backfire.
I completely agree with this approach. It is just that starting is the hardest part and if do something every day at least a little bit, you will make it a lot easier to start on command. That is one of the advantages. Normally thinking about alignment for 1 minute is not causing an internal war I expect. I think having this goal is good, and there is no conflict necessarily with what you are talking about. I think it is best if both are combined.
The popular evo-psych idea “self-deception for the purpose of other-deception” is largely a myth.
It is very real in my experience. In hindsight, I have caught myself many times. Maybe I mean something different than you. Motivated early stopping would fall into this for me, when you are sort of not aware/suppressing to become aware of that you are doing this. Which I think is the default. I am very sure I have observed my mind suppressing further thought once it early stopped with some ridiculous justification.
I stress introspection so much because it’s ~futile to make novel object-level progress without a deep familiarity/understanding of the tools you use for the task.
I think this is wrong. I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did. Though I agree that I could probably have done better with introspection.
I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did.
More specifically, I mean progress wrt some long-term goal like AI alignment, altruism, factory farming, etc. Here, I think most ways of thinking about the problem are wildly off-target bc motivations get distorted by social incentives. Whereas goals in narrow games like “win at chess” or “solve a math problem” are less prone to this, so introspection is much less important.
Well, I am talking about creating games, not playing them if that was unclear. I think that is significantly harder than making games. It took over a thousand hours of practice to get good. I think AI alignment is a lot harder, but I think the same pattern applies to some extent. For example, asking the question “What will this project look like if it goes really well is a good question.” Why well when John asked this question to a bunch of people he got good results. I have not thought about why you get good results, but asking this question. But I am pretty sure I could understand it better, and that is likely to be useful, compared to not understanding. But clearly, you can get benefits even when you don’t understand.
Most of the time when you are applying a technique you will just be applying the technique. You will normally not retrieve all of the knowledge of why this technique works before using it. And it works fine. The knowledge about why the technique is mostly useful for refining the technique is my guess. However, applying the refined technique does not require retrieving the knowledge. In fact, you might often forget the knowledge but not the refined technique, i.e. the procedural knowledge.
I completely agree with this approach. It is just that starting is the hardest part and if do something every day at least a little bit, you will make it a lot easier to start on command. That is one of the advantages. Normally thinking about alignment for 1 minute is not causing an internal war I expect. I think having this goal is good, and there is no conflict necessarily with what you are talking about. I think it is best if both are combined.
It is very real in my experience. In hindsight, I have caught myself many times. Maybe I mean something different than you. Motivated early stopping would fall into this for me, when you are sort of not aware/suppressing to become aware of that you are doing this. Which I think is the default. I am very sure I have observed my mind suppressing further thought once it early stopped with some ridiculous justification.
I think this is wrong. I was very very terrible at introspection just 2 years ago. Yet I did manage to learn how to make very good games, without really introspecting for years later about why the things work that I did. Though I agree that I could probably have done better with introspection.
More specifically, I mean progress wrt some long-term goal like AI alignment, altruism, factory farming, etc. Here, I think most ways of thinking about the problem are wildly off-target bc motivations get distorted by social incentives. Whereas goals in narrow games like “win at chess” or “solve a math problem” are less prone to this, so introspection is much less important.
Well, I am talking about creating games, not playing them if that was unclear. I think that is significantly harder than making games. It took over a thousand hours of practice to get good. I think AI alignment is a lot harder, but I think the same pattern applies to some extent. For example, asking the question “What will this project look like if it goes really well is a good question.” Why well when John asked this question to a bunch of people he got good results. I have not thought about why you get good results, but asking this question. But I am pretty sure I could understand it better, and that is likely to be useful, compared to not understanding. But clearly, you can get benefits even when you don’t understand.
Most of the time when you are applying a technique you will just be applying the technique. You will normally not retrieve all of the knowledge of why this technique works before using it. And it works fine. The knowledge about why the technique is mostly useful for refining the technique is my guess. However, applying the refined technique does not require retrieving the knowledge. In fact, you might often forget the knowledge but not the refined technique, i.e. the procedural knowledge.