moridinamael(Matt Freeman)
It has helped me to realize that the policies I implement to govern my behavior are implicit hypotheses about my behavior. The general form is “I predict that installing policy X will produce behavior Y.” In the past when my imposed self-regulatory policies have failed I have viewed it as a personal failure, a failure of the will, my failure to adhere to the policy. It helped a lot to realize that my hypothesis was merely wrong. I became free to update. I started learning from my failures.
It’s funny, I am totally sympathetic to everything you wrote here, yet all I can think is, “my daily life is chock full of people incapable of grappling with trolley problems or discussing torture concretely, why are you trying to make LessWrong more like real life?”
After seeing a number of rather gloomy posts on the site in the last few days, I feel a need to point out that problems that we don’t currently know how to solve always look impossible. A smart guy once pointed out how silly it was the Lord Kelvin claimed “The influence of animal or vegetable life on matter is infinitely beyond the range of any scientific inquiry hitherto entered on.” Kelvin just didn’t know how to do it. That’s fine. Deciding it’s a Hard Problem just sort of throws up mental blocks to finding potential obvious solutions.
Maybe alignment will seem really easy in retrospect. Maybe it’s the sort of thing that requires only two small insights that we don’t currently have. Maybe we already have all the insights we need and somebody just needs to connect them together in a non-obvious way. Maybe somebody has already had the key idea, and just thought to themselves, no, it can’t be that simple! (I actually sort of viscerally suspect that the lynchpin of alignment will turn out to be something really dumb and easy that we’ve simply overlooked, and not something like Special Relativity.) Everything seems hard in advance, and we’ve spent far more effort as a civilization studying asphalt than we have alignment. We’ve tried almost nothing so far.
In the same way that we have an existence-proof of AGI (humans existing) we also have a highly suggestive example of something that looks a lot like alignment (humans existing and often choosing not to do heroin), except probably not robust to infinite capability increase, blah blah.
The “probabilistic mainline path” always looks really grim when success depends on innovations and inventions you don’t currently know how to do. Nobody knows what probability to put on obtaining such innovations in advance. If you asked me ten years ago I would have put the odds of SpaceX Starship existing at like 2%, probably even after thinking really hard about it.
I have found that the more I use my simulation of HPMOR!Quirrell for advice, the harder it is to shut him up. As with any mental discipline, thinking in particular modes wears thought-grooves into your brain’s hardware, and before you know it you’ve performed an irreversible self-modification. Consequently, I would definitely recommend that anybody attempting to supplant their own personality (for lack of a better phrasing) with a model of some idealized reasoner try to make sure that the idealized reasoner shares your values as thoroughly as possible.
Took it. Nothing like a census/survey to make you feel like part of a community.
FYI
Advisory: RANDOM.ORG will be temporarily unavailable on Sunday 2012-11-04 due to a system upgrade.
Also, in retrospect, I misremembered my own age. It’s been … a busy year.
It seems so obvious to me that the benefits of preschool would wear off after a short number of years that I feel like I must be missing something. How could it be otherwise, given the current system? This is all completely setting aside the developmental limitations of small children.
Let’s take two kids, Jamie and Alex. Pretend that there are no developmental limitations on children’s brains and that they can be taught to read equally well at ages 3, 4, 5, and 6.
Alex starts preschool at age 3 and they can read at a 1st grade level by the time they enter Kindergarten.
Jamie does not do any preschool and cannot read at all when they enter Kindergarten.
By the end of Kindergarten, Alex can read slightly better than 1st grade level, but not a lot better, since the curriculum hasn’t been challenging. It’s basically been a rehash of what they already can do. Jamie can read at the expected grade level by the end of Kindergarten.
By the end of first grade, no accommodations have been made for the fact that Alex is a slightly advanced reader. Both kids are given essentially the same pool of books to read. Alex has not skipped a grade or put in an some secret fast-track program for kids who went to preschool, because this does not exist. So by the end of first grade, they can read about equally well. Maybe Alex reads slightly better, but since no real pressure is being put on this advantage that would cause it to compound rather than diminish, it naturally diminishes until both students are at the same level.
Acting as though anything else would happen doesn’t make sense to me. It’s not like each year a child spends in school exerts some kind of Education Force on their brain which accrues generalized scholastic ability. Kindergartners are taught kindergarten level math and reading skills; kids entering kindergarten who already possess these skills only benefit until the other kids catch up.
So IMO the problem isn’t the preschool “doesn’t do anything”. The problem is that the system as it stands doesn’t actually utilize the potential advantage of preschool. We are pretty far away from a system that would do so; such a system would need to be one that tailors the specific educational content to the specific child.
My four year old can read pretty well and can write well enough that you can puzzle out what he’s trying to communicate. But there is no expectation that he’s going to skip kindergarten because of this. So in what sense could this ever be a long-term academic advantage?
You’ve made me understand the root of one of my own dissatisfactions with the current system. If I look through my post history and roughly group my posts into bins based on how I would summarize them, this is what I see:
Silly posts in the HP:MOR threads: ~ +20 karma
Posts of mine having little content except to express agreement with other high-karma posts: ~ +10 karma
Important information or technical corrections in serious discussions: ~ +1 karma
Posts which I try to say something technical which I retrospectively realize were poorly worded but could have been clarified if someone pointed out an issue instead of just downvoting: ~ −5 karma
Perhaps I exaggerate slightly but my point is that if I were to formulate a posting strategy aimed at obtaining karma, then I would avoid saying anything technical or discussing anything serious and stick to applause lights and fluff.
On top of this, I tend to watch how the karma of my most recent comments behaves, and so I notice that, for example, a comment might have +5 upvotes and −3 downvotes, with no replies. This is just baffling to me. Was there something wrong with the post that three people noticed? Were the three separate things wrong with it? Was it just a response to the tone? What about the upvotes, is it being upvoted because of the witticism at the end, or because of the technical content in the middle? My point is something like Slashdot has a system where things are voted “funny” or “insightful” would be infinitely more useful.
[Please read the OP before voting. Special voting rules apply.]
Buying a lottery ticket every now and then is not irrational. Unless you have thoroughly optimized the conversion of every dollar you own into utility-yielding investments and expenses, the exposure to large positive tail risk netted by spending a few dollars on lottery tickets can still be rational.
Phrased another way, when you buy a lottery ticket you aren’t buying an investment, you’re buying a possibility that is not available otherwise.
I’m a male LWer with an infant daughter. I’d like to request some specific advice on avoiding the common failure modes.
As to your first paragraph, one of the more liberating things about internalizing the Sequences and inculcating myself with Less Wrong memes is that it I’ve come to hold very few opinions strongly. When you move yourself to a sufficiently morally-relativistic framework you stop identifying with your opinions. Instead of saying there are things you “just have to believe” I would say there are things that it is instrumentally rational to behave as if you believe.
Regarding your second paragraph: I find that I simply get into fewer arguments. Because I have let go of most of the opinions that typically weight people and make them respond at an emotional level to being contradicted, every conversation that includes a disagreement becomes a joint truth-seeking venture. Instead of arguments, I have “discussions.” If the other party in the discussion is not interested in truth seeking but is instead interested in being right, I just stop humoring them and change the subject. If they are someone I can be honest with, I will point out that they seem to have an irrational bias regarding the topic in question.
It seems like your first mistake was getting involved in a theological debate. Science is flawed, but “religion” doesn’t even have enough predictive power to be falsified. I would step back at least one level and urge you to ask yourself what your objectives are in participating in such a confused discussion in the first place. I myself have indulged in totally stupid internet arguments which I can only attribute to a sort of perverse pique that strikes me at times, but I generally admit that I’m already failing to be rational just by participating in such.
The company I work for specializes in buying neglected oil and gas properties from giant oil companies and, basically, performing archaeology to figure out what the giant oil company was doing so we can do it better. It’s quite fun poring through a well file that goes back a hundred years, starting with hand-written oil rates, margin notes and initials by untold numbers of engineers and field hands. The solution to what you think is an engineering problem—say, anomalously high gas production in the historic record—can sometimes be literally a historical phenomenon, like, everybody was flat-out lying about about gas production during that time period because of something called the “80′s Gas Wars.”
It sounds like the mystery engineer worked at a very old probably borderline economical plant. The fact that the plant was so maligned by management can be read as an indication that it was a very marginal enterprise. One gets the sense that some vice president flipped a coin regarding the decision of whether to do the debottlenecking project or just shut the plant down and save on manpower. I say this to make the following point: infrastructure decay happens to infrastructure that kind of sucks. In a way, this is part of the circle of life, the work of the invisible hand, or whatever. While this plant was falling into decrepitude, other newer ones with better technology were being built according to more modern best practices by different companies. If this plant had mattered, I don’t think this would have been as likely to happen.
I am certain that someone will point out to me an example of a high-value industrial work that was allowed to fall into decrepitude and neglect, but I do think there’s a causal relationship between neglect and uselessness. The reason I am holding forth on this is that I have heard so many dozens of engineers whine about how their projects were mistreated by management and always silently ached to reply, “So what? Your project sucked. Management is trying to optimally allocate resources. Grow up.” There, I feel better.
When I was about seven years old I was playing an imagination game which gradually became unusually vivid and complex. In the fantasy I was delivering some kind of technical presentation depicting what looked like electron microscope images of small tube structures and I kept using the word ‘microhydraulics’ in my speech.
After a long time I snapped out of the fantasy and just sort of stood transfixed, introspecting on what I had just experienced. I realized it was far more immersive than my typical fantasies, even at the time, and I had the distinct sense that a lot of time had passed since I was actually aware of my surroundings. Further, I did not typically fantasize about giving technical presentations, I didn’t know what the word ‘microhydraulics’ meant and I probably had never seen an image from an electron microscope..
Decades later I have earned my PhD in a topic concerning modeling of fluid flow through micro- and nanoscopic media. I have given many technical presentations on the topic. Many of these technical presentations have included SEM images of such microscale flow features. Occasionally I will use the word “microhydraulics” in these talks, even though it is not really proper technical jargon … because no matter how old I get I can’t shake the intuition that I was seeing my future in that childhood game.
Of course, it is a heck of a lot more likely that this weird childhood experience subtly affected my interests over the course of my life and led me to eventually study the field that I studied. However, if you were to actually look at my life story, it would look a lot more like a series of random coincidences which effectively chose my research area for me, so at this point I like to just playfully pretend that what I experienced was a glitch in the matrix.
They exist, but it’s like this: you walk into the store. To your left, there are forty pink dresses and onesies with Cutest Princess or somesuch printed on them. To your right, there are forty blue onesies and overall combos, often with anthropomorphic male animals printed on them. In the middle, there are three yellow or green onesies.
On top of that, well-meaning relatives send us boxes of the pink dresses.
When I dress her, I avoid the overtly feminine outfits. But then I worry that I’m committing an entirely new mistake. I imagine my daughter telling me how confused she felt that her father seemed reluctant to cast her as a girl. “Did you wish I was a boy, Daddy?” There don’t seem to be many trivially obvious correct choices in parenting.
Before now, it wasn’t immediately obvious that SC2 is a game that can be played superhumanly well without anything that looks like long-term planning or counterfactual reasoning. The way humans play it relies on a combination of past experience, narrow skills, and “what-if” mental simulation of the opponent. Building a superhuman SC2 agent out of nothing more than LSTM units indicates that you can completely do away with planning, even when the action space is very large, even when the state space is VERY large, even when the possibilities are combinatorially enormous. Yes, humans can get good at SC2 with much less than 200 years of time played (although those humans are usually studying the replays of other masters to bootstrap their understanding) but I think it’s worthwhile to focus on the inverse of this observation: that a sophisticated problem domain which looks like it ought to require planning and model-based counterfactual reasoning actually requires no such thing. What other problem domains seem like they ought to require planning and counterfactual reasoning, but can probably be conquered with nothing more advanced than a deep LSTM network?
(I haven’t seen anyone bother to compute an estimate of the size of the state-space of SC2 relative to, for example, Go or Chess, and I’m not sure if there’s even a coherent way to go about it.)
A: (punches B) Slug bug!
B: (immediately punches A back, roughly equally hard)
A: Hey! You don’t get to hit me back. That’s the rules.
B: I understand. However, I was actually playing Iterated Prisoner’s Dilemma. And so were you, by the way. Furthermore, I wasn’t aware that we were playing Slug Bug, so my prior has to be that you were actually just demonstrating or testing your physical dominance over me.
A: We’re friends! We’ve known each other for forty years! You’re godfather to my children! I’m married to your sister! Why would I be demonstrating physical dominance over you?
B: I guess this is one of those better-safe-than-sorry situations. I actually do trust you to lay down your life for mine, but I don’t trust you to perfectly and continuously control your human status impulses. You stepped over a line, I checked you on it. Now we can proceed with neither of us wondering if that punch had any hidden implications to our relationship. By the way, now that you’ve announced we’re playing Slug Bug, I’m game for it.
A: But I wasn’t playing this “status-regulation-prisoner’s-dilemma” that you’re describing, I was playing Slug Bug. Under my rules, I should just punch you a second time!
B: Do what you must. Be aware that I will just punch you back, again.
A: You’re so insecure! I bodily carried you through a minefield and on a separate occasion threw myself onto a grenade to save you.
B: I consider the fact that you obviously love and cherish me to be completely separate from the issue of instinctive dominance and status behavior. Put it this way: if you expect the dynamics our relationship, and its base of sacred trust, to permit you to punch me for no apparent reason, then you must symmetrically expect our relationship to permit me applying a very mild, exactly proportionate corrective measure. In fact, I’m far more concerned that you’re surprised and angry that I punched you back, than I was concerned that you punched me in the first place. Your concern implies that you did assume an asymmetry in our positions. I’ll put it even more clearly: You can’t punch somebody, expect them not to make a big deal of it, and then proceed to make a big deal of it when they punch you, regardless of what game you think you’re playing in your own mind.
A: (punches B) Slug bug! You weren’t paying attention.
B: Dammit.
Meta-college-advice-advice: Treat college advice, whether online or in person, as a literary genre. Deconstruct it.
What are the probable motives of the advice-giver? Might they be trying to enlist Effective Altruists, future FAI researchers, political allies, or future low-wage employees, graduate students, code monkeys, CAD monkeys, or foot soldiers? Might they be misrepresenting the opportunities for promotion out of said entry level grunt positions? Basically, are they trying to manipulate labor out of you, or are they offering you friendly advice out of the kindness of their generous hearts? If they obviously do have an agenda, can that be okay, and why?
Does the advice-giver seem to share your values? Related to but distinct from the first bullet point, you will have a deluge of advice aimed at convincing you to make the “best” choice, from fifty different perspectives, rarely with any of the advice-givers pausing to reflect that when they say “best” they of course mean “best” according to their own complex criteria.
Does the advice-giver seem to be inextricably trapped in a doldrums of hindsight bias and/or anchoring? For example, do they seem to think that it’s patently, blindingly obvious that everyone should specialize in plant cell membrane chemistry, and maybe if you want to take a risk you could specialize in animal cell membrane chemistry, but you would have to be an utter fool to do something like engineering? I find this attitude to be incredibly common even in extremely intelligent people. I think from the inside it feels like, “I wouldn’t be doing quantum spectroscopy if quantum spectroscopy weren’t the correct choice.”
As far as I can tell there is now literally a giant machine somewhere pumping out propaganda aimed at convincing young people to do start-ups. Rather than saying my opinion on this, I will simply urge that you apply the same level of scrutiny to this breathless, manic material as you apply to your dad’s pitch to go to law school.
When there is a real wolf free among the sheep, it will be too late to cry wolf. The time to cry wolf is when you see the wolf staring at you and licking its fangs from the treeline, not when it is eating you. The time when you feel comfortable expressing your anxieties will be long after it is too late. It will always feel like crying wolf, until the moment just before you are turned into paperclips. This is the obverse side of the There Is No Fire Alarm for AGI coin.
It sounds like an old house, based on the structural abnormalities. Is it possible that there is a gas leak? One prime candidate for the “haunted house” phenomenon is hallucinations brought on by exposure to carbon monoxide.
I think it worth pointing out that it’s very difficult to remove selective perception and confirmation bias here. For example, if I experienced the “dropped key” scenario you described, I would simply assume that the key had slid or been kicked out of the room, and then someone else had later found it and set it down somewhere off the ground. However, if I believed that I lived in a Haunted House, the first hypothesis that would spring to mind would be that the key’s teleportation was part of the haunting. So when you say such incidents have not occurred since you lived in that house, I believe that you think that, but it’s likely that you just don’t view incidents outside the house as existing in the same magisterium, in some sense.
Additionally, I happened to be pondering today why people seem to take Aumann’s Agreement Theorem so literally around here. It is a mathematical idealization. We’re not talking about the mathematics of particle physics, we’re talking about the cognition and interactions of humans, who only barely do reasoning in the first place and barely succeed in communicating even the simplest concepts without signal loss.
Was taking it, and it crashed with a “This webpage is not available” error.
I sometimes worry that ideas are prematurely rejected because they are not guaranteed to work, rather than because they are guaranteed not to work. In the end it might turn out that zero ideas are actually guaranteed to work and thus we are left with an assortment of not guaranteed to work ideas which are underdeveloped because some possible failure mode was found and thus the idea was abandoned early.