The general does not exist, there are only specifics.
If I have a thought in my head, “Texans like their guns”, that thought got there from a finite amount of specific interactions. Maybe I heard a joke about texans. Maybe my family is from texas. Maybe I hear a lot about it on the news.
“People don’t like it when you cut them off mid sentence”. Which people?
At a local meetup we do a thing called encounter groups, and one rule of encounter groups is “there is no ‘the group’, just individual people”. Having conversations in that mode has been incredibly helpful to realize that, in fact, there is no “the group”.
(Less a reply and more just related)
I often think a sentence like, “I want to have a really big brain!“. What would that actually look like?
Not experiencing fear or worry when encountering new math.
Really quick to determine what I’m most curious about.
Not having my head hurt when I’m thinking hard, and generally not feeling much “cognitive strain”.
Be able to fill in the vague and general impressions with the concrete examples that originally created them.
Doing a hammers and nails scan when I encounter new ideas.
Having a clear, quickly accessible understanding of the “proof chains” of ideas, as well as the “motivation chains”.
I don’t need to know all the proofs or motivations, but I do have a clear sense of what I understand myself, and what I’ve outsourced.
Instead of feeling “generally confused” by things of just “not getting them”, I always have concrete, “This doesn’t make sense because BLANK” expressions that allow me to move forward.
What are the barriers to having really high “knowledge work output”?
I’m not capable of “being productive on arbitrary tasks”. One winter break I made a plan to apply for all the small $100 essay scholarships people were always telling me no one applied for. After two days of sheer misery, I had to admit to myself that I wasn’t able be productive on a task that involved making up bullshit opinions about topics I didn’t care about.
Conviction is important. From experiments with TAPs and a recent bout of meditation, it seems like when I bail on an intention, on some level I am no longer convinced the intention is a good idea/what I actually want to do. Strong conviction feels like confidence all the way up in the fact that this task/project is the right thing to spend your time on.
There’s probably a lot in the vein of have good chemistry: sleep well, eat well, get exercise.
One of the more mysterious quantities seems to be “cognitive effort”. Sometimes thinking hard feel like it hurts my brain. This post has a lot of advice in that regard.
I’ve previously hypothesized that the a huge chunk of painful brain fog is the experience of thinking at a problem, but not actually engaging with it. (similar to how Mark Forster has posited that the resistance one feels to a given task is proportional to how many times it has been rejected)
Having the rest of your life together and time boxing your work is insanely important for reducing the frequency with which your brains promotes “unrelated” thoughts to your consciousness (if there’s important stuff that isn’t getting done, and you haven’t convinced yourself that it will be handled adequately, your mind’s tendency is to keep it in a loop).
I’ve got a feeling that there’s a large amount of gains in the 5-second level. I would be super interested in seeing anyone’s thoughts or writings on the 5-second level of doing better work and avoiding cognitive fatigue.
Yay self-study! Positive reinforcement!
I was just thinking about this earlier today while re-reading a similar point by stuart armstrong.
AFI worry: A human-in-the-loop AI that only takes actions that get human approval (and whose expected outcomes have human approval) hits big problems when the context the AI is acting in is a very different context from where our values were trained.
Is there any way around this besides simulating people having their values re-organized given the new environment? Is this what CEV is about?
Weirdness that comes from reflection:
In this frame, I can self-reflect on a given circuit and ask, “Does this circuit actually push me towards what I think is good?” When doing this, I’ll be using some more meta/higher-order circuits (concepts I’ve built up over time about what a “good” brain looks like) but I’ll also be using lower level circuits, and I might even end up using the evaluated circuit itself in this evaluation process.
Sometimes this reflection process will go smooth. Sometimes it won’t. But one takeaway/claim is you have this complex roundabout process for re-evaluating your values when some circuits begin to think that other circuits have diverged from “good”.
Because of this ability to reflect and change, it seems correct to say that “I value things conditional on my environment” (where environment has a lot of flex, it could be as small as your work space, or as broad as “any existing human culture”).
Example. Let’s say there was literally no scarcity for survival goods (food water etc). It seems like a HUGE chunk of my values and morals are built up inferences and solutions to resource allocation problems. If resource scarcity was magically no longer a problem, much of my values have lost their connection to reality. From what I’ve seen so far of my own self-reflection process, it seems likely that overtime I would come to reorganize my values in such a post-scarcity world. I’ve also currently got no clue what that reorganization would look like.
Current beliefs about how human value works: various thoughts and actions can produce a “reward” signal in the brain. I also have lots of predictive circuits that fire when they anticipate a “reward” signal is coming as a result of what just happened. The predictive circuits have been trained to use the patterns of my environment to predict when the “reward” signal is coming.
Getting an “actual reward” and a predictive circuit firing will both be experienced as something “good”. Because of this, predictive circuits can not only track “actual reward” but also the activation of other predictive circuits. (So far this is basically “there’s terminal and instrumental values, and they are experienced as roughly the same thing”)
The predictive circuits are all doing some “learning process” to keep their firing correlated to what they’re tracking. However, the “quality” of this learning can vary drastically. Some circuits are more “hardwired” than others, and less able to update when they begin to become uncorrelated from what they are tracking. Some are caught in interesting feedback loops with other circuits, such that you have to update multiple circuits simultaneously, or in a particular order.
Thought every thing that feels “good” feels good because at some point or another it was tracking the base “reward” signal, it won’t always be a good idea to think of the “reward” signal as the thing you value.
Say you have a circuit that tracks a proxy of your base “reward”. If something happens in your brain such that this circuit ceases to update, you basically value this proxy terminally.
Said another way, I don’t have a nice clean ontological line between terminal values and instrumental values. The less valuable a predictive circuit, the more “terminal” the value it represents.
My thoughts are similar to Paul’s comment.
You might be interested in the difference between high-level languages and Domain Specific Languages (DSL). High-level languages are intended to be general purpose languages that abstract away some of the nitty gritty details of lower levels (C doesn’t have to think about registers while assembly does, python doesn’t have to think about memory management while C does). DSLs are languages specially designed for certain problem domains. Stan and nile are examples of DSLs for probabilistic programming and displaying graphics, respectively. You seem to be suggesting a DSL for AI as opposed to a generic higher-level language.
To generalize, it seems like DSLs and higher-level languages are useful to make doing something you already know how to do more intuitive/simple/straight-forward. Making AGI and AFI are things we don’t know how to do. We could likely make a DSL that would allow you to think more fluently about existing AI concepts, and it’s possible that might allow you to reach various insights quicker. But you still have to do the work.
Wanted to note that I really enjoyed this and thought it was quite lucid. A great example of how thinking about possible detailed implementations of a mind help you dodge a lot of the fudging that with various philosophical puzzles.
I’m going to put meditation on pause, but still keep my protected 30 min morning time. I’m going to swap in “Thinking, introspecting, and dwelling on how I relate to other people.”
The decisions comes from a few things:
How I relate to people is one of the most frequent subjects of mind-wandering during meditation.
I’m currently in a relationship that has made me a lot more aware of various habits of behavior and thinking that I have.
I feel well poised to make a lot of progress on that front.
When I mind notice I’m mind wandering, I try to follow, “Why did my attention go from the my breathe to this topic?” I frequently find that I pretty deeply feel that spending time on relations would benefit me more than the meditation (ala this comment).
I’m going to give this a one-month trial period. Expect an update March 31, 2019
Often I have an idea three times in various forms before it makes it to territory of, “Well thought out idea that I’m actually acting upon and having good stuff come from it.”
My default, I follow a pattern of, “semi-randomly expose myself to lots of ideas, not worry a lot about screening for repetitive stuff, let the most salient ideas at any given moment float up to receive tid-bits of conscious thought, then forget about them till the next semi-random event triggers it being thought about.”
I’d be interested if there was a better protocol for, “This thing I’ve encountered seems extra important/interesting, let me dwell on it more and more intentionally integrate it into my thinking/”
It’s been 112 days on this habit.
5 days have been missed, none happened consecutively. “Don’t skip twice” has been a good star to follow.
For the past month and a half I’ve felt a good deal more mind-wandering than in the first month and a half. This seemed to be related to not giving as much attention to why I am doing this, forgetting the exciting motivation, and treating it as, “routine”. This past week has been a slight uptick in attention. We’ll see what happens.
Would you buy the claim you can “Be more right and get less wrong”? (asked because I feel like I’m pointing to the same thing as the first bullet, but the first bullet is not phrased super well)
On the question of “does understanding your mind make you fail less often”, I notice that there are 3+ cases that immediately jump to mind that match, “I didn’t fail because I learned more about my mind”. Do you think a lot of those cases I didn’t fail for reasons other than understanding my mind, or do you expect that I’m racking up new-different failures as a result of understanding my mind more?
On post rationality, I just now read a bit more, and my reaction was, “Wait, wasn’t that a key piece of rationality in the first place?” I’m interested to see if I’ve secretly always been of a post-rationalist persuasion, or if I’m not seeing the main angle of post-rationality.
Cool. There’s nothing I currently want transfered. This was more a forward thinking thought. I’ll pm you if it comes up.
Thanks! As of now, is there any tool for migrating comments from one post to another? I just revised my approach to this sequence in a way that I expect to reduce the need for that (more clearly separating structure and content posts), but I’d imagine it could be useful.
1st plan was that this and the prev posts where “outlines of the main ideas I want to cover in however many posts, to be written later”. When there was enough detail of, “Here’s a discrete post/worthy idea I’ve decided I want to cover” I would then create new post which would be the flushed out outline of the idea. This would then be edited and revised to be a normal human readable content post.
You made me notice that I didn’t have a clear sense of how I wanted comments to work. If there’s discussion of the ideas/content outlined in this post, would I want to / have to port that to the actual post I eventually make?
Current plan is to clearly differentiate “structure” and “content” posts. This is a structure post, and I’d like comments to be about, “What do you think of this chunk of idea space being an intro to rationality?“. Content posts will be, “What do you think of this idea/content/how I wrote about it?”
From Gwern’s about page:
I personally believe that one should think Less Wrong and act Long Now, if you follow me.
Possibly my favorite catch-phrase ever :) What do I think is hiding there?
Think Less Wrong
Self anthropology- “Why do you believe what you believe?”
Hugging the Query and not sinking into confused questions
Litany of Tarski
Notice your confusion - “Either the story is false or you model is wrong”
Act Long Now
Cultivate habits and practice routines that seem small / trivial on a day/week/month timeline, but will result in you being superhuman in 10 years.
Build abstractions where you are acutely aware of where it leaks, and have good reason to believe that leak does not affect the most important work you are using this abstraction for.
What things trigger “Man, it sure would be useful helpful if I had data on XYZ from the past 8 years”? Start tracking that.