CynicalOptimist

Karma: 79

CynicalOptimist Nov 18, 2016, 12:15 AM
3 points
in reply to: benelliott’s comment on: Confidence levels inside and outside an argument
This is good, but I feel like we’d better represent human psychology if we said:

Most people don’t make a distinction between the concepts of “x has probability <0.1%” and “x is impossible”.

I say this because I think there’s an important difference between the times when people have a precise meaning in mind, which they’ve expressed poorly, and the times when people’s actual concepts are vague and fuzzy. (Often, people don’t realise how fuzzy their concepts are).

CynicalOptimist Nov 17, 2016, 8:29 PM
0 points
in reply to: someonewrongonthenet’s comment on: Timeless Identity
This seems to me like an orthogonal question. (A question that can be entirely extricated and separated from the cryonics question).

You’re talking about whether you are a valuable enough individual that you can justify resources being spent on maintaining your existence. That’s a question that can be asked just as easily even if you have no concept of cryonics. For instance: if your life depends on getting medical treatment that costs a million dollars, is it worth it? Or should you prefer that the money be spent on saving other lives more efficiently?

(Incidentally, i know that utilitarianism generally favours the second option. But I would never blame anyone for choosing the first option if the money was offered to them.)

I would accept an end to my existence if it allowed everyone else on earth to live for as long as they wished, and experience an existentially fulfilling form of happiness. I wouldn’t accept an end to my existence if it allowed one stranger to enjoy an ice cream. There are scenarios where I would think it was worth using resources to maintain my existence, and scenarios where I would accept that the resources should be used differently. I think this is true when we consider cryonics, and equally true if we don’t.

The cryonics question is quite different.

For the sake of argument, I’ll assume that you’re alive and that you intend to keep on living, for at least the next 5 years. I’ll assume that If you experienced a life-threatening situation tomorrow, and someone was able to intervene medically and grant you (at least) 5 more years of life, then you would want them to.

There are many different life-threatening scenarios, and many different possible interventions. But for decision making purposes, you could probably group them into “interventions which extend my life in a meaningful way” and interventions that don’t. For instance, an intervention that kept your body alive but left you completely brain-dead would probably go in the second category. Coronary bypass surgery would probably go in the first.

The cryonics question here is simply: “If a doctor offered to freeze you, then revive you 50 years later” would you put this in the same category as other “life-saving” interventions? Would you consider it an extension of your life, in the same way as a heart transplant would be? And would you value it similarly in your considerations?

And of course, we can ask the same question for a different intervention, where you are frozen, then scanned, then recreated years later in one (or more) simulations.

CynicalOptimist Nov 17, 2016, 8:08 PM
0 points
in reply to: TheOtherDave’s comment on: Timeless Identity
I think I’ve got a good response for this one.

My non-episodic memory contains the “facts” that Buffy the Vampire Slayer was one of the best television shows that was ever made, and the Pink Floyd aren’t an interesting band. My boyfriend’s non-episodic memory contains the facts that Buffy was boring, unoriginal, and repetetive (and that Pink Floyd’s music is trancendentally good).

Objectively, these are opinions, not facts. But we experience them as facts. If I want to preserve my sense of identity, then I would need to retain the facts that were in my non-episodic memory. More than that, I would also lose my sense of self if I gained contradictory memories. I would need to have my non-episodic memories and not have the facts from my boyfriend’s memory.

That’s the reason why “off the shelf” doesn’t sound suitable in this context.

CynicalOptimist Nov 17, 2016, 7:26 PM
0 points
on: The Optimizer’s Curse and How to Beat It
Very interesting. I’m going to try my hand at a short summary:

Assume that you have a number of different options you can choose, that you want to estimate the value of each option and you have to make your best guess as to which option is most valuable. In step one, you generate individual estimates using whatever procedure you think is best. In step 2 you make the final decision, by choosing the option that had the highest estimate in step one.

The point is: even if you have unbiased procedures for creating the individual estimates in step one (ie procedures that are equally likely to overestimate as to underestimate) biases will still be introduced in step 2, when you’re looking at the list of all the different estimates. Specifically, the biases are that the highest estimate(s) are more likely to be overestimates, and the lowest estimate(s) are more likely to be underestimates.

CynicalOptimist Nov 17, 2016, 6:28 PM
0 points
in reply to: Vladimir_Nesov’s comment on: The Optimizer’s Curse and How to Beat It
Well in some circumstances, this kind of reasoning would actually change the decision you make. For example, you might have one option with a high estimate and very high confidence, and another option with an even higher estimate, but lower confidence. After applying the approach described in the article, those two options might end up switching position in the rankings.

BUT: Most of the time, I don’t think this approach will make you choose a different option. If all other factors are equal, then you’ll probably still pick the option that has the highest expected value. I think that what we learn from this article is more about something else: It’s about understanding that the final result will probably be lower than your supposedly “unbiased” estimate. And when you understand that, you can budget accordingly.

CynicalOptimist Nov 17, 2016, 5:27 PM
0 points
in reply to: Brickman’s comment on: The Optimizer’s Curse and How to Beat It
I think there’s some value in that observation that “the all 45 thing makes it feel like a trick”. I believe that’s a big part of why this feels like a paradox.

If you have a box with the numbers “60” and “20″ as described above, then I can see two main ways that you could interpret the numbers:

A: The number of coins in this box was drawn from a probability distribution with a mean of 60, and a range of 20.

B: The number of coins in this box was drawn from an unknown probability distribution. Our best estimate of the number of coins in this box is 60, based on certain information that we have available. We are certain that the actual value is within 20 gold coins of this.

With regards to understanding the example, and understanding how to apply the kind of Bayesian reasoning that the article recommends, it’s important to understand that the example was based on B. And in real life, B describes situations that we’re far more likely to encounter.

With regards to understanding human psychology, human biases, and why this feels like a paradox, it’s important to understand that we instinctively tend towards “A”. I don’t know if all humans would tend to think in terms of A rather than B, but I suspect the bias applies widely amongst people who’ve studied any kind of formal probability. “A” is much closer to the kind of questions that would be set as exercises in a probability class.

CynicalOptimist Nov 10, 2016, 12:34 PM
0 points
in reply to: TheAncientGeek’s comment on: Magical Categories
I think that RobbBB has already done a great job of responding to this, but I’d like to have a try at it too. I’d like to explore the math/morality analogy a bit more. I think I can make a better comparison.

Math is an enormous field of study. Even if we limited our concept of “math” to drawing graphs of mathematical functions, we would still have an enormous range of different kinds of functions: Hyperbolic, exponential, polynomial, all the trigonometric functions, etc. etc.

Instead of comparing math to morality, I think it’s more illustrative to compare math to the wider topic of “value-driven-behaviour”.

An intelligent creature could have all sorts of different values. Even within the realm of modern, western, democratic morality we still disagree about whether it is just and propper to execute murderers. We disagree about the extent to which a state is obligated to protect its citizens and provide a safety net. We disagree about the importance of honesty, of freedom vs. safety, freedom of speech vs. protection from hate speech.

If you look at the wider world, and at cultures through history, you’ll find a much wider range of moralities. People who thought it was not just permitted, but morally required that they enslave people, restrict the freedoms of their own families, and execute people for religious transgressions.

You might think that these are all better or worse approximations of the “one true morality”, and that a superintelligence could work out what that true morality is. But we don’t think so. We believe that these are different moralities. Fundamentally, these people have different values.

Then we can step further out, and look at the “insane” value systems that a person could hold. Perhaps we could believe that all people are so flawed that they must be killed. Or we could believe that no one should ever be allowed to die, and so we extend life indefinitely, even for people in agony. Or we might believe everyone should be lobotomised for our own safety.

And then there are the truly inhuman value systems: the paperclip maximisers, the prime pebble sorters, and the baby eaters. The idea is that a superintelligence could comprehend any and all of these. It would be able to optimise for any one of them, and foresee results and possible consequences for all of them. The question is: which one would it actually use?

A superintelligence might be able to understand all of human math and more besides, but we wouldn’t build one to simply “do all of maths”. We would build it with a particular goal and purpose in mind. For instance (to pick an arbitrary example) we might need it to create graphs of Hyperbolic functions. It’s a bad example, I know. But I hope it serves to help make the point.

Likewise, we would want the intelligence to adopt a specific set of values. Perhaps we would want them to be modern, western, democratic liberal values.

I wouldn’t expect a superintelligence to start generating Hyperbolic functions, despite the fact that it’s smart enough to do so. The AI would have no reason to start doing that particular task. It might be smart enough to work out that that’s what we want of course, but that doesn’t mean it’ll do it (unless we’ve already solved the problem of getting them to do “what humans want it to do”.) If we want Hyperbolic functions, we’ll have to program the machine with enough focus to make it do that.

Likewise, a computer could have any arbitrary utility function, any arbitrary set of values. We can’t make sure that a computer has the “right” values unless we know how to clearly define the values we want.

With Hyperbolic functions, it’s relatively easy to describe exactly, unambiguously, what we want. But morality is much harder to pin down.

CynicalOptimist Nov 9, 2016, 12:29 AM
0 points
in reply to: TheOtherDave’s comment on: The Hidden Complexity of Wishes
But if you do care about your wishes being fulfilled safely, then safety will be one of the things that you want, and so you will get it.

So long as your preferences are coherent, stable, and self-consistent then you should be fine. If you care about something that’s relevant to the wish then it will be incorporated into the wish. If you don’t care about something then it may not be incorporated into the wish, but you shouldn’t mind that: because it’s something you don’t care about.

Unfortunately, people’s preferences often aren’t coherent and stable. For instance an alcoholic may throw away a bottle of wine because they don’t want to be tempted by it. Right now, they don’t want their future selves to drink it. And yet they know that their future selves might have different priorities.

Is this the sort of thing you were concerned about?

CynicalOptimist Nov 8, 2016, 11:46 PM
0 points
in reply to: Eric_1’s comment on: The Hidden Complexity of Wishes
I like this style of reasoning.

Rather than taking some arbitrary definition of black boxes and then arguing about whether they apply, you’ve recognised that a phrase can be understood in many ways, and we should use the word in whatever way most helps us in this discussion. That’s exactly the sort of rationality technique we should be learning.

A different way of thinking about it though, is that we can remove the confusing term altogether. Rather than defining the term “black box”, we can try to remember why it was originally used, and look for another way to express the intended concept.

In this case, I’d say the point was: “Sometimes, we will use a tool expecting to get one result, and instead we will get a completely different, unexpected result. Often we can explain these results later. They may even have been predictable in advance, and yet they weren’t predicted.”

Computer programming is especially prime to this. The computer will faithfully execute the instructions that you gave it, but those instructions might not have the net result that you wanted.

CynicalOptimist Nov 8, 2016, 11:30 PM
0 points
in reply to: Eric_1’s comment on: The Hidden Complexity of Wishes
“if the Pump could just be made to sense the proper (implied) parameters.”

You’re right, this would be an essential step. I’d say the main point of the post was to talk about the importance, and especially the difficulty, of achieving this.

Re optimisation for use: remember that this involves a certain amount of trial and error. In the case of dangerous technologies like explosives, firearms, or high speed vehicles, the process can often involve human beings dying, usually in the “error” part of trial and error.

If the technology in question was a super-intelligent AI, smart enough to fool us and engineer whatever outcome best matched its utility function? Then potentially we could find ourselves unable to fix the “error”.

Please excuse the cheesy line, but sometimes you can’t put the genie back in the bottle.

Re the workings of the human brain? I have to admit that I don’t know the meaning of ceteris paribus, but I think that the brain mostly works by pattern recognition. In a “burning house” scenario, people would mostly contemplate the options that they thought were “normal” for the situation, or that they had previously imagined, heard about, or seen on TV

Generating a lot of different options and then comparing them for expected utility isn’t the sort of thing that humans do naturally. It’s the sort of behaviour that we have to be trained for, if you want us to apply it.

CynicalOptimist Nov 8, 2016, 11:12 PM
2 points
in reply to: J_Thomas’s comment on: The Hidden Complexity of Wishes
I agree, just because something MIGHT backfire, it doesn’t mean we automatically shouldn’t try it. We should weigh up the potential benefits and the potential costs as best we can predict them, along with our best guesses about the likelihood of each.

In this example, of course, the lessons we learn about “genies” are supposed to be applied to artificial intelligences.

One of the central concepts that Eliezer tries to express about AI is that when we get an AI that’s as smart as humans, we will very quickly get an AI that’s very much smarter than humans. At that point, the AI can probably trick us into letting it loose, and it may be able to devise a plan to achieve almost anything.

In this scenario, the potential costs are almost unlimited. And the probability is hard to work out. Therefore figuring out the best way to program it is very very important.

Because that’s a genie… {CSI sunglasses moment} … that we can’t put back in the bottle.

CynicalOptimist Nov 8, 2016, 10:53 PM
2 points
in reply to: Nick_Tarleton’s comment on: The Hidden Complexity of Wishes
I see where you’re coming from on this one.

I’d only add this: if a genie is to be capable of granting this wish, it would need to know what your judgements were. It would need to understand them, at least as well as you do. This pretty much resolves to the same problem that Eliezer already discussed.

To create such a genie, you would either need to explain to the genie how you would feel about every possible circumstance, or you would need to program the genie so as to be able to correctly figure it out. Both of these tasks are probably a lot harder than they sound.

CynicalOptimist Nov 7, 2016, 11:57 PM
−1 points
0
in reply to: JessRiedel’s comment on: Magical Categories
Can’t agree with this enough.

CynicalOptimist Nov 5, 2016, 9:32 AM
1 point
in reply to: Multiheaded’s comment on: Ethics Notes
Alternate answer:

If the Kremlin publicly announces a policy, saying that they may reward some soldiers who disobey orders in a nuclear scenario? Then this raises the odds that a Russian official will refuse to launch a nuke—even when they have evidence that enemy nukes have already been fired on Russia.

(So far, so good. However...)

The problem is that it doesn’t just raise the odds of disobedience, it also raises the perceived odds as well. ie it will make Americans think that they have a better chance of launching a first strike and “getting away with it”.

A publically announced policy like this would have weakened the USSR’s nuclear deterrent. Arguably, this raises everyone’s chances of dying in a nuclear war, even the Americans.

CynicalOptimist Nov 5, 2016, 9:20 AM
3 points
in reply to: elharo’s comment on: Ethics Notes
It may be an uncommon scenario, but it’s the scenario that’s under discussion. We’re talking about situations where a soldier has orders to do one thing, and believes that moral or tactical considerations require them to do something else—and we’re asking what ethical injunctions should apply in that scenario.

To be fair, Jubilee wasn’t very specific about that.

CynicalOptimist Oct 7, 2016, 2:29 AM
3 points
in reply to: CCC’s comment on: Humans in Funny Suits
Yup! I agree completely.

If you were modeling an octopus-based sentient species, for the purposes of writing some interesting fiction, then this would be a nice detail to add.

CynicalOptimist Oct 7, 2016, 2:06 AM
0 points
in reply to: gjm’s comment on: Talking Snakes: A Cautionary Tale
Thank you. :)

CynicalOptimist Aug 19, 2016, 3:18 PM
0 points
in reply to: Decius’s comment on: Conservation of Expected Evidence
I believe the idea was to ask “hypothetically, if I found out that this hypothesis was true, how much new information would that give me?”

You’ll have two or more hypotheses, and one of them is the one that would (hypothetically) give you the least amount of new information. The one that would give you the least amount of new information should be considered the “simplest” hypothesis. (assuming a certain definition of “simplest”, and a certain definition of “information”)

CynicalOptimist 12 May 2016 21:57 UTC
0 points
in reply to: Günther_Greindl’s comment on: When (Not) To Use Probabilities
This is excellent advice.

I’d like to add though, that the original phrase was “algorithms that make use of gut feelings… ”. This isn’t the same as saying “a policy of always submitting to your gut feelings”.

I’m picturing a decision tree here: something that tells you how to behave when your gut feeling is “I’m utterly convinced” {Act on the feeling immediately}, vs how you might act if you had feelings of “vague unease” {continue cautiously, delay taking any steps that constitute a major commitment, while you try to identify the source of the unease}. Your algorithm might also involve assessing the reliability of your gut feeling; experience and reason might allow you to know that your gut is very reliable in certain matters, and much less reliable in others.

The details of the algorithm are up for debate of course. For the purposes of this discussion, i place no importance on the details of the algorithm i described. The point is just that these procedures are helpful for rational thinking, they aren’t numerical procedures, and a numerical procedure wouldn’t automatically be better just because it’s numerical.

CynicalOptimist 10 May 2016 21:14 UTC
0 points
in reply to: CCC’s comment on: Talking Snakes: A Cautionary Tale
I think this is the basis of good Business Analysis. A field I’m intending to move into.

It’s the very essence of “Hold off on proposing solutions”.