gRR

Karma: 523

gRR 1 May 2012 12:10 UTC
78 points
on: Rationality Quotes May 2012

Once upon a time, there was a man who was riding in a horse drawn carriage and traveling to go take care of some affairs; and in the carriage there was also a very big suitcase. He told the driver to of the carriage to drive non-stop and the horse ran extremely fast.
Along the road, there was an old man who saw them and asked, “Sir, you seem anxious, where do you need to go?”
The man in the carriage then replied in a loud voice, “I need to go to the state of Chu.” The old man heard and laughing he smiled and said, “You are going the wrong way. The state of Chu is in the south, how come you are going to to the north?”
“That’s alright,” The man in the carriage then said, “Can you not see? My horse runs very fast.”
“Your horse is great, but your path is incorrect.”
“It’s no problem, my carriage is new, it was made just last month.”
“Your carriage is brand new, but this is not the road one takes to get to Chu.”
“Old Uncle, you don’t know,” and the man in the carriage pointed to the suitcase in the back and said, “In that suitcase there’s alot of money. No matter how long the road is, I am not afraid.”
“You have lots of money, but do not forget, The direction which you are going is wrong. I can see, you should go back the direction which you came from.”
The man in the carriage heard this and irritated said, “I have already been traveling for ten days, how can you tell me to go back from where I came?” He then pointed at the carriage driver and said, “Take a look, he is very young, and he drives very well, you needn’t worry. Goodbye!”
And then he told the driver to drive forward, and the horse ran even faster.

--Chinese Tale

gRR 3 Apr 2012 2:10 UTC
26 points
on: Harry Potter and the Methods of Rationality discussion thread, part 13, chapter 81
Hypothesis: the Source of Magic is an AI with the goal to work in the way (magical) people really believe it should work. Or maybe, to make the world work in the way (magical) people really believe it should work. The strength of belief appears to be important, so a strong belief can override weak ones. On the other hand, when something is already “generally known” to work in a certain way, this is a very strong belief.

Examples:
1. Broomsticks work by Aristotelian physics [because it was what people believed when the broomsticks were invented, and now people just know (=believe really strongly) that’s how broomsticks should behave]
2. Spell names and laws [inventors create spells by finding sounds they believe should work. When spells become known, they stabilize in that form]
3. Potions Law
4. Ritual magic [people really believe in sacrifices and not getting something for nothing]
5. Ghosts (and afterlife?) [effects of religious beliefs]
6. Harry’s partial transfiguration [very strong belief, finds a loophole to not be in conflict with existing strong beliefs of other people]
Magic doesn’t make sense to Harry because it now reflects lots of ad hoc rules and beliefs accumulated in centuries. Wizards and witches believe them from childhood. [No wonder they are half-insane.]

Interestingly, this hypothesis implies that Dumbledore’s narrative causality may actually work—people do believe in stories.

gRR 3 Apr 2012 12:52 UTC
23 points
on: Evidence for the orthogonality thesis
A possible intuition may come from computer games. In principle, a game world can be arbitrarily complex, and the goals (“winning conditions”) are specified by the programmers, so they can be anything at all. For example, the world may be Medieval Fantasy setting, while the goal may be to invent, craft, take from NPCs by force, and otherwise collect as many paperclips as possible. If an external incentive is provided for people to play the game (e.g., they get real-world money for each paperclip), the players will essentially become paperclip maximizers within the game world.

gRR 26 Mar 2012 2:55 UTC
23 points
on: Harry Potter and the Methods of Rationality discussion thread, part 12
1. Dumbledore believes that Voldemort is at large.
2. And that he’s probably responsible for Hermione’s troubles.
3. And that he can possess people.
4. And that Quirrell is under heavy suspicion, both as a Defense Professor and directly in this case.
5. And Dumbledore still looks for Tom Riddle elsewhere.
6. And he doesn’t hold the Idiot Ball (because no one in this fic does).
I notice I am confused.
What links here?
- gRR's comment on Harry Potter and the Methods of Rationality discussion thread, part 16, chapter 85 by FAWS (19 Apr 2012 12:44 UTC; 3 points)

gRR 17 Mar 2012 10:57 UTC
23 points
on: Muehlhauser-Goertzel Dialogue, Part 1
It’s strange that people say the arguments for Big Scary Idea are not written anywhere. The argument seems to be simple and direct:
1. Hard takeoff will make AI god-powerful very quickly.
2. During hard takeoff, the AI’s utility=goals=values=what-it-optimizes-for will solidify (when AI understand its own theory and self-modify correspondingly), and even if it was changeable before, it will be unchangeable forever since.
3. Unless the AI goals embody every single value important for humans and are otherwise just right in every respect, the results of using god powers to optimize for these goals will be horrible.
4. Human values are not a natural category, there’s little to no chance that AI will converge on them by itself, unless specifically and precisely programmed.
The only really speculative step is step 1. But if you already believe in singularity and hard foom, then the argument should be unrefutable...

gRR 15 Mar 2012 11:59 UTC
19 points
on: Cult impressions of Less Wrong/Singularity Institute
I’m here for only a couple of months, and I didn’t have any impression of cultishness. I saw only a circle of friends doing a thing together, and very enthusiastic about it.

What I also did see (and still do) is specific people just sometimes being slightly crazy, in a nice way. As in: Eliezer’s threatment of MWI. Or way too serious fear of weird acausal dangers that fall out of currently best decision theories.
Note: this impression is not because of craziness of the ideas, but because of taking them too seriously too early. However, the relevant posts always have sane critical comments, heavily upvoted.

I’m slightly more alarmed by posts like How would you stop Moore’s Law?. I mean, seriously thinking of AI dangers is good. Seriously considering nuking Intel’s fabs in order to stop the dangers is… not good.

gRR 3 May 2012 23:32 UTC
18 points
on: Urgent: Subjects needed for rationality measure development
Sorry for being paranoid, but this is the internet, and you’re a newcomer, and you’re asking for lots of personal data for unclear reasons (Phone? Full name? Whatever for?), and the question “Which sequences have you read at least 75% of?” is not well-defined without a list of sequence names...

gRR 21 Apr 2012 22:08 UTC
16 points
in reply to: fubarobfusco’s comment on: Against the Bottom Line

If you come up with a candidate “bottom line” and then explore arguments for and against it, and sometimes end up rejecting it, then it wasn’t really a bottom line — your algorithm hadn’t actually terminated.

Oh. That makes sense. So it’s the bottom line only if I write it and refuse to change it forever after. Or, if it is the belief on which I actually act in the end, if it was all a part of a decision-making process.

Guess that’s what everybody was telling me… feeling stupid now.

gRR 19 Apr 2012 0:26 UTC
16 points
on: How can we get more and better LW contrarians?
I would prefer an increase in ‘question’ (problem) posts, as opposed to ‘statement’ (solution) posts, contrarian or no.

gRR 23 May 2012 19:01 UTC
13 points
on: Problematic Problems for TDT
The problems look like a kind of an anti-Prisoner’s Dilemma. An agent plays against an opponent, and gets a reward iff they played differently. Then any agent playing against itself is screwed.

gRR 9 Mar 2012 12:03 UTC
13 points
on: [SEQ RERUN] Amazing Breakthrough Day: April 1st
I like this.

SECRETS OF THE PHRAOS! Mysterious Relation Between Pyramids Height and Base Discovered
PURIFY YOURSELF DAILY: Tiny Evil Demons Cause Diseases Everywhere—Priests Call For Water Rituals
UNIVERSAL LOVE? Everybody Is Attracted To Everybody Else Says Cambridge Don
THE ABACUS: Mesopotamians Present Computing Device—Performance Up To 200 Milliflops
HUFFLEPUFFS INSIDE: Autopsies Show Human Bodies Contain Hard Sharpenable Objects

Anthropic Reasoning by CDT in Newcomb’s Problem

gRR14 Mar 2012 0:44 UTC

9 points

36 comments1 min readLW link

gRR 8 May 2012 20:36 UTC
9 points
in reply to: TheOtherDave’s comment on: A wild theist platonist appears, to ask about the path
Well then, you could also multiply gods by constants and add them together, producing a vector space over a divine basis.

Grounding morality works straightforwardly, I think: God said thou must not kill, love thy neighbour, etc.

gRR 4 May 2012 20:16 UTC
9 points
on: Rationality and Winning
It would be more correct to say that “Winning as defined by general society norms” is not a strong measure of rationality. “Actual Winning”, as defined by the agent’s own total values, certainly must be.

gRR 18 Apr 2012 18:04 UTC
9 points
on: Open Thread, April 16 − 30, 2012
A question for rationalist parents (and anyone else who has ideas): are there good child-accessible rational arguments for why do right?

Me: Please do X.
Child: No.
Me: You know it’s the right thing to do.
Child: Yes.
Me: Well?
Child: I don’t want to.
Me: ???

Against the Bottom Line

gRR21 Apr 2012 10:20 UTC

8 points

30 comments1 min readLW link

gRR 7 Apr 2012 21:34 UTC
8 points
in reply to: gwern’s comment on: Rationality Quotes April 2012
It would be interesting to calculate the total utility of an author wantonly murdering a universally beloved character. May turn out to be quite a crime...

gRR 5 Apr 2012 19:58 UTC
8 points
in reply to: kilobug’s comment on: Harry Potter and the Methods of Rationality discussion thread, part 14, chapter 82
Actually, the Word of Eliezer didn’t kill this hypothesis yet. He only said nothing in the story represents either FAI or UFAI, in the allegorical sense. But a real FAI/UFAI can still be there.

gRR 15 Mar 2012 14:19 UTC
8 points
in reply to: wedrifid’s comment on: Cult impressions of Less Wrong/Singularity Institute
Yes to well bonded. People here seem to understand each other far better than average on the net, and it is immediately apparent.

Enthusiastic is a wrong word, I suppose. I meant, sure of doing a good thing, happy to be doing it, etc, not in the sense of applauding and cheering.

gRR 1 May 2012 22:48 UTC
7 points
in reply to: NancyLebovitz’s comment on: Rationality Quotes May 2012
I first saw the story in “School in Carmarthen”, which I would absolutely recommend to everyone, except it’s in Russian. I thought there should probably be an English translation of the Chinese tale, so I googled it up by keywords.

The tale is apparently the origin story behind a common Chinese idiom that literally translates as “south house north rut”, and which means acting in a way that defeats one’s purpose.

gRR

An­thropic Rea­son­ing by CDT in New­comb’s Problem

Against the Bot­tom Line

Anthropic Reasoning by CDT in Newcomb’s Problem

Against the Bottom Line