Rationalists should win
I hate to see this clever statement to be taken out of context and being reinterpreted as a moralizing slogan.
If you trace it back, this was originally a desideratum on rational thinking, not some general moral imperative.
It plainly says that if your supposedly “rational” strategy leads to a demonstrably inferior solution or gets beaten by some stupider looking agent, then the strategy must be reevaluated as you have no right to call it rational anymore.
Q: How do you convince a singularitarian to eat shit?
A: Declare eating shit shock level 5
Chess World Champions are sometimes notoriously superstitious, you can still rely on the consistency of their chess moves.
Interesting. Eliezer took some X years to recognize that even “normal looking” persons can be quick on the uptake? ;)
My best guess is that clickiness has something to do with failure to compartmentalize—missing, or failing to use, the mental gear that lets human beings believe two contradictory things at the same time. Clicky people would tend to be people who take all of their beliefs at face value.
I guess it has a bit deeper explanation than that. I think clickiness happens if two people managed to build very similar mental models and they are ready to manipulate and modify models incrementally. Once the models are roughly in sync, it takes very little time to communicate and just slight hints can create the right change in the conversation partner’s model, if he is ready to update.
I think a lot of us has been trained hard to stop model building at certain points. There is definitely a personal difference between people on how much do they care about taboos the society imposes on them which can result in mental red lights: “Don’t continue building that model! It’s dangerous!” This is what I think Eliezer’s notion of “compartmentalization” refers to.
A lot of intelligent people have much less brakes and generally used to model building and do this uninhibitedly, can maintain several models and have fun doing that.
But in general, most people are lazy model builder, they build their models once and stick to it, or just find rationalizations to cut down on “mental effort” of generating and incrementally updating models.
However, I don’t think that brakes are the only reason people don’t click. If I just don’t know how to build the right model (miss the know-how, experience, etc.) I won’t click regardless.
For example I am not a musician and if I found it hard to have conversations with experienced musicians over music. Musicians among themselves can build very quickly models of musical concepts and click with each other. I can try to be maximally open minded, I won’t manage to click. I simply fail the necessary skills to build the required models.
I think it is mostly hopeless trying to teach rationality to most people.
For example, both of my parents studied Math in university and still have a very firm grip of the fundamentals.
I just got a phone call yesterday from my father in Germany saying: “We saw in the news, that a German tourist couple got killed in a shooting in San Francisco. Will you avoid going out after dark?” When I tried to explain that I won’t update my risk estimates based on any such singular event, he seemed to listen to and understand formally what I said. Anyhow, he was completely unimpressed, finishing the conversation in an even more worried tone: “I see, but you will take care, won’t you?”
IMO, the views that rational analysis and manipulation in social context (esp. with regard to mating) is immoral or dehumanizing is based on observations that a lot of people who consciously employ such techniques often have the wrong objective function.
Consider this analogue situation: If you raise children, you definitely do a lot of rational thinking about their needs, long term interests and try help them to develop, be safe, etc. This requires a lot of objective considerations, prioritizing, even conscious manipulations on several different levels. Nobody would say that this is wrong, dehumanizing or out of place. The reason that this is intuitively accepted is that you probably do this for the right reason: in the long term interest of your children. If you have the right objective function, it is not just fine, it is required.
I think that other social interactions should not be different in this respect. You should consciously employ techniques, objectively analyze and manipulate situations, but your objective function should include the interests of your peers as well. They will sense if you genuinely care about them: Even if you manipulate them, they will be still thankful later if it happened in their long term interests.
You don’t become a “manipulative bastard” just because you are manipulative, but if you are also a bastard.
OMEGA: If you pay me just one penny, I’ll replace your 80% chance of living for 10^(10^10) years, with a 79.99992% chance of living 10^(10^(10^10)) years
HUMAN: That sounds like an awful lot of time. Would you mind to write it as a decimal number
OMEGA: Here it is… Of course don’t expect to read this number in less than 10 ^ 9999999990 years.
HUMAN: Nevermind… So It’s such a mind boggling amount of time. If I would get bored or otherwise distressed, loose my lust for life. Am I allowed to kill myself?
OMEGA: Not really. If I’d allow that and assuming that the probability of killing yourself would be 0.000000001 in 10^10 years, then it would be almost sure that you kill yourself by the end of 10^(10^(10^10)) years
HUMAN: This sounds depressing. So my decision has the potential to confining me to grillions of years of suffering, if I’d lost my lust for life.
OMEGA: OK, I see your point, I also offer you some additional drugs to make you happy whenever you would have any distress. I also promise you to modify your brain that you will never even wish to kill yourself during these few eons.
HUMAN: Sounds great,, but I also enjoy your company very much, can I hope you to entertain me from time to time with bets like this?
OMEGA: Yes, BUT: you will never be able to enter a bet again that has a nonzero chance of survival. Otherwise you will enter too many bets over your lifetime so that it would become inevitable that you die long before the end of the lifetime.
HUMAN: Hmmm, interesting… I think I could still live with that. In this case I take your offer.
OMEGA: But wait! There’s another special offer, and you won’t even have to pay a penny for this one—this one is free! That’s right, I’m offering to exponentiate your lifespan again, to 10^(10^(10^10,000,000,000)) years! Now, I’ll have to multiply your probability of survival by 99.9999% again, but really, what’s that compared to the nigh-incomprehensible increase in your expected lifespan?
HUMAN: Interesting. However that sounds like a bit irrealistic to me. How long do we know each other?
OMEGA: Why do you ask?
HUMAN: I already determined to agree to your original offer. Loosing those eons even at 0.0001% chance sounds quite risky to me. I just want to check whether I can trust you enough to expect that risk to pay off?
OMEGA: What do you mean?
HUMAN: I need some Bayesian evidence that you are really capable of keeping your promise.
OMEGA: What do you expect me to do?
HUMAN: I am in a generous mood. I think 0.000001% of that 10^(10^(10^10))) years of supporting me, answering my questions, providing company would be enough to convince me that you mean your offer seriously enough such that it is worth that 0.0001% risk of loosing the remaining 99.999999% of those eons.
OMEGA: Nice try tricking me. If I granted your wish, you’d get 10^(10^999999990) years with a potential prolongation to 10^(10^(10^10)) years.
HUMAN: Come on, I just try to be rational, how could I trust you without enough Bayesian evidence? I’d risk gazillions of eons, for a dubious payoff. That requires extraordinary amount of evidence.
OMEGA: This stinks. I don’t do anything like that. Either you trust me 100% or not. Your choice.
HUMAN: No problem. Even with 100% certainty in your capabilities, I am not 100% sure about mine. You see: I’ve got only those puny neurons and only 10^11 of them and you wish me to make an educated choice on 10^(10^(10000000000)) years? We have already found several caveats: possibility suicide, unhappiness, bayesian evidence etc. that had to be resolved. How can I be sure that that’s all? If there is some other catch that I could not think of and then I’ll have to live with my decision for 10^(10^(1000000000)) years? Certainly the probability of overlooking something that would spoil those years is high enough not to risk taking the offer. Anyways what is the probability that you come up with an even more convincing offer during a period of 10^1000000000 years?
Whenever I read about “weight of experience”, “quantum goo”, “existentness” etc. I can’t keep myself of also thinking of “vital spark”, “phlogiston”, “ether” and other similar stuff… And it somehow spoils the whole fun...
In the history of mankind, hard looking (meta-)physical dilemmas were much more often resolved by means of elimination rather than by introduction of new “essences”. The moral of the history of physics so far is that relativity typically trumps absoluteness in the long run.
For example, I would not be surprised at all, if it turned out that experienced Born probabilities would not be absolute, but would depend on some reference frame (in a very high dimensional space) just like the experience of time, speed, mass, etc. depends on the relativistic frame of reference.
In European bars or nightclubs, if (relatively) attractive girls ask strangers for drinks or dishes, then it typically means they are doing it professionally.
There is even a special phrase “consume girl” meaning that the girl’s job is to lure clueless customers into buying expensive drinks for them for a cut of the profit. The surest sign of being a “consume girl” is that they typically don’t consume what they ask for.
It’s all about money, and has nothing to do with social games, whatsoever. They are not spoiled brats, but trained for this job.
I am not sure how common is this “profession” in the US, but in Europe it’s relatively common.
Omega seems to run into some very fundamental credibility problem:
Let us assume the premise of the OP, that lifetime can be equated with discrete computational operations. Furthermore, also assume that the universe (space-time/mulltiverse/whatever) can be modeled result of a computation of n operations (let us say for simplicity n=10^100, we could also assume 10^^100 or any finite number, we will just need a bit more iterations of offers then).
… after some accepted offers … :
OMEGA: … I’ll replace your p% chance of living for 10^n years, with a 0.9999999p% chance of living 10^(10^n) years...
AGENT: Sounds nice, but I already know, what I would do first with 10^n years.
OMEGA: ???
AGENT: I will simulate my previous universe up to the current point. Inclusive this conversation.
OMEGA: What for?
AGENT: Maybe I am a nostalgic type. But even if I would not be, Given so much computational resources, the probability that I would not do it accidentally would be quite negligible.
OMEGA: Yes, but you could do even more simulations if you would take my next offer. AGENT: Good point, but how can I tell that this conversation is not already taking place in that simulation? Whatever you would tell me (even if you are outside my universe, your intervention in my world would only add a few bits of extra information to those n bits), would not even change the information content of my universe by a single order of magnitude, so it would definitely end up among my countless emulations. I don’t see any conceivable way of you being able to convince me, that I am not the one emulated by a version of myself that already gained the resources in question, especially that I am determined to run these simulations just for FUN.
We already have these buttons on LessWrong… ;)
This will mark the first successful paper-clip-maximizer-unboxing-experiment in human history… ;)
Working on papers submitted to peer-reviewed scientific journals is not marketing but research.
If SIAI wants to build some credibility then it needs some publications in scientific journals. Doing so could help to ensure further funding and development of actual implementations.
I think that it is a very good idea to first formulate and publish the theoretical basis for the work they intend to do, rather than just saying: we need money to develop component X of our friendly AI.
Of course a possible outcome will be that the scientific community will deem the research shallow, unoriginal or unrealistic to implement. However, it is necessary to publish the ideas before they can be reviewed.
So my take on this is that SIAI is merely asking for a chance to demonstrate their skills rather than for blind commitment.
“Drug” was just a catchy phrase for omega’s guarantee to cure you out from any psychological issues the could cause you any prolonged distress.
You could insist that it is entirely impossible that you’d need it.
Would not it be a bit overconfident to make any statements on what is possible to some insanely complex and alien future self of you over a period of time which is measured by a number (in years) that takes billion to the power of billions of your current lifetime just to read?
The statement It all adds up to normality. as far as I can trace back, was just a simple practical recognition that the MW interpretation of quantum mechanics was consistent with our everyday experience.
A simple, objective, true statement without any imperatives or any overreaching philosophical implications.
Unfortunately, over the time, it became a popular mantra to be repeated every time someone expresses some inconvenient sounding ontological statement that does not fit someone else’s warm cozy Star Trek world view.
“It could be, but don’ think about that. It always adds up to normality,. anyways… *
Isn’t it?
There is a technique which I did not find in the above lists, but found most useful for myself (I’d give it a strong +7).
I will call it Leave some tasty bits for next morning meaning that at the end of the working day I leave some task very well prepared which I am excited about to start to work the next day, something which is:
easy to start with (finished with bulk of the tedious details)
gives instant gratification
a good start to get into less pleasant tasks
The idea is that it is in general hard to get into the “work mode”, but once one gets the right momentum it is much easier to continue. So often I found that it is better not to finish some easy but exciting stuff at the end of the day, but leave it for next morning in the hope of setting up my whole day.
The wrong tactic is to do the tasty bits first and leave the tedious details to start the next day with. That way, it is sure that I will spend the whole morning doing something unproductive instead.
This is irrelevant. Human players make mistakes. The question is whether being superstitious makes them make more mistakes.
Shock-level-bragging is so 2003… ;) Still, in my opinion, this post contains some extremely interesting unconventional intuition, which seems to be way underrated.
In the long run, this whole mathematical multiverse idea has the potential to become much less insubstantial than it may look on superficial inspection.
There are quite a few problems with it though. For example the reliance on minimum description length does not feel like the right approach to the probability problem at that level. It may turn out to be eventually, but generally, the perceived probabilities don’t come from a conscious decision to care about some abstract (and uncomputable!) complexity measure (like MDL)
No! We experience certain probabilities, because something is built into the very nature of our physics (or rather meta- or multi-physics). So, even if MDL turns out to be at the core, it must be a derived consequence, rather than just being pulled out of the hat as in the OP.
The most essential clue in the puzzle could be, if we’d manage to understand how this glue works that connects processes with phenomenologically equivalent or similar information processing structures. I can see Barbour’s derivation of the general relativity as an interesting analogy. But it’s obviously much harder to argue about general (approximate) isomorphisms of causality networks than measuring the similarity of mass distributions of three dimensional spaces.
Nevertheless I think the intuition multiverse ideas provide could inspire speculations in extremely exciting directions: For example: Is it conceivable that the symmetries we experience in the physical laws are plainly consequences of the natural symmetries of this observer-gluing process itself?
It is pretty obvious that Alcor must avoid even the slightest suspicion of helping (or even encouraging) you to commit suicide.
This would be an extremely slippery slope that they really have to avoid in order to prevent exposing themselves to a lot of unjustified attacks.
I also have a practical advise to those who try to improve their social skills especially in the dating space.
Learn some real partner dance!
By “partner dance” I mean something that requires real partnering skills like salsa, swing or tango, (rather than hip-hop or techno, etc. that have no real partner dancing scene and culture).
Although I don’t do it for pick-up or dating reasons (I am married with children), I have been dancing salsa for a few years and I recognized a lot of positive effects:
Great workout (much more fun than anything else I tried)
Learned to behave much more relaxed and self-confident in the company of women.
Improved body-awareness, posture, gracefulness, etc.
Taught me how to make bodily contact in a gentle and enjoyable way and without being intrusive or offensive and feel completely natural about it.
Made me more aware of dressing and hygiene.
A form of social meditation: living in the presence, enjoying the moment by concentrating not on oneself but on someone else and the music.
But first of all: It is an awful lot of fun. Imagine that you to go to work or to a conference in a city you’ve never been before and at the end of the day you can be almost sure that you will find a place where there are several people of the opposite sex wait to chat with you a bit, spend a few minutes in your arms without any significant risks or hurt feelings. This was my experience at every single place I visited: I counted that within the last two years I danced in 4 different countries 15 different cities and over 30 different venues.
I am neither really young, nor good looking. It’s also fair to say that I am extremely introverted, geeky and clumsy. Still, I managed to have more fun than I’ve ever expected. For example two months ago I went to a salsa event and was sitting around a bit between two songs when a beautiful woman came up to me and asked me to dance. I never say no, but maybe I should have, because she was so much above my level: it was almost embarrassing… Later in the night I told my wife that I danced with that amazing woman. She looked at her and told me: “Good for you, you just danced with Liz Lira” (Turned out my wife went to her workshop, during the day) Indeed: I looked her up in the net and recognized: 5X salsa world champion..., but what a nice person!