# Tetraspace

Karma: 522
• 80,000 Hours’ job board lets you filter by city. As of the time of writing, roles in their AI Safety & Policy tag are 61112 San Francisco, 16112 London, 35112 other (including remote).

• There are about 8 billion people, so your 24,000 QALYs should be 24,000,000.

• I don’t mean to say that it’s additional reason to respect him as an authority or accept his communication norms above what you would have done for other reasons (and I don’t think people particularly are here), just that it’s the meaning of that jokey aside.

• Maybe you got into trouble for talking about that because you are rude and presumptive?

I think this is just a nod to how he’s literally Roko, for whom googling “Roko simulation” gives a Wikipedia article on what happened last time.

• What, I wonder, shall such an AGI end up “thinking” about us?

IMO: “Oh look, undefended atoms!” (Well, not in that format. But maybe you get the picture.)

You kind of mix together two notions of irrationality:

I think only the first one is really deserving of the name “irrationality”. I want what I want, and if what I want is a very complicated thing that takes into account my emotions, well, so be it. Humans might be bad at getting what they want, they might be mistaken a lot of the time about what they want and constantly step on their own toes, but there’s no objective reason why they shouldn’t want that.

Still, when up against a superintelligence, I think that both value being fragile and humans being bad at getting what they want count against humans getting anything they want out of the interaction:

• Superintelligences are good at getting what they want (this is really what it means to be a superintelligence)

• Superintelligences will have whatever goal they have, and I don’t think that there’s any reason why this goal would be anything to do with what humans want (the orthogonality thesis; the goals that a superintelligence has are orthogonal to how good it is at achieving them)

This together adds up to a superintelligence sees humans using resources that it could be using for something else (and it would want them to use them for something else, not just what the humans are trying to do but more, because it has its own goals), and because it’s good at getting what it wants it gets those resources, which is very unfortunate for the humans.

• Arbital gives a distinction between “logical decision theory” and “functional decision theory” as:

• Logical decision theories are a class of decision theories that have a logical counterfactual (vs. the causal counterfactual that CDT has and the evidential counterfactual EDT has).

• Functional decision theory is the type of logical decision theory where the logical counterfactual is fully specified, and correctly gives the logical consequences of “decision function X outputs action A”.

More recently, I’ve seen in Decision theory does not imply that we get to have nice things:

• Logical decision theory is the decision theory where the logical counterfactual is fully specified.

• Functional decision theory is the incomplete variant of logical decision theory where the logical consequences of “decision function X outputs action A” have to be provided by the setup of the thought experiment.

Any preferences? How have you been using it?

• Further to it being legally considered murder, tricky plans to get around this are things that appear to the state like possibly a tricky plan to get around murder, and result in an autopsy which at best and only if the cryonics organisation cooperates leaves one sitting around warm for over a day with no chance of cryoprotectant perfusion later.

• For some inspiration, put both memes side by side and listen to Landsailor. (The mechanism by which one listens to it, in turn, is also complex. I love civilisation.)

• The essay What Motivated Rescuers During the Holocaust is on Lesswrong under the title Research: Rescuers during the Holocaust—it was renamed because all of the essay titles in Curiosity are questions, which I just noticed now and is cute. I found it via the URL lesswrong.com/​​2018/​​rescue, which is listed in the back of the book.

The bystander effect is an explanation of the whole story:

• Because of the bystander effect, most people weren’t rescuers during the Holocaust, even though that was obviously the morally correct thing to do; they were in a large group of people who could have intervened but didn’t.

• The standard way to break the bystander effect is by pointing out a single individual in the crowd to intervene, which is effectively what happened to the people who became rescuers by circumstances that forced them into action.

• Why would you wait until ? It seems like at any time the expected payoff will be , which is strictly decreasing with .

# [Question] Is there a “co­her­ent de­ci­sions im­ply con­sis­tent util­ities”-style ar­gu­ment for non-lex­i­co­graphic prefer­ences?

29 Jun 2021 19:14 UTC
4 points
• The real number 0.20 isn’t a probability, it’s just the same odds but written in a different way to make it possible to multiply (specifically you want some odds product * such that A:B * C:D = AC:BD). You are right about how you would convert the odds into a probability at the end.