TimFreeman

Karma: 751

TimFreeman Jul 10, 2011, 6:13 PM
0 points
in reply to: Raw_Power’s comment on: Dark Arts: Schopenhauer wrote The Book on How To Troll

When dealing with trolls, whether on the Internet or in Real Life, no matter how absolutely damn sure you are of your point, you have no time to unravel their bullshit for what it is, and if you try it you will only bore your audience and exhaust their patience. Debates aren’t battles of truth: there’s publishing papers and articles for that. Debates are battles of status.

I agree. There’s also the scenario where you’re talking to a reasonable person for the purpose of figuring out the truth better than either of you could do alone. That’s useful, and it’s important to be able to distinguish that from debating with trolls for the purpose of gaining status. Trolls can be recognized by how often they use rhetoric that obviously isn’t truth-seeking, and Schopenhauer is very good for that.

Well, actually, on the Internet you never gain status by debating with trolls. Even if I win an argument, I lose status to the extent my behavior justifies the conclusion “Tim wastes time posting to (LessWrong|SlashDot|whatever) instead of doing anything useful.”

My ability to identify and stonewall trolls varies. Sometimes I catch them saying something silly and refuse to continue unless they correct themselves, and that stops the time-waste pretty quickly. Sometimes I do three-strikes-and-your-out, and the time-waste stops reasonably soon. Sometimes it takes me a long time to figure out if they’re a troll, especially if they’re hinting that they know something worthwhile. I wish I had a more stable rule of thumb for doing this right. Any suggestions?

TimFreeman Jul 9, 2011, 12:41 AM
0 points
on: The Threat of Cryonics
Terror Management seems to explain the reactions to cryonics pretty well. I’ve only skimmed the OP enough to want to trot out the standard explanation, so I may have missed something, but so far as I can tell the Historical Death Meme and Terror Management make the same predictions.

It is in fact absolutely unacceptable, from a simple humanitarian perspective, that something as nebulous as the HDM—however artistic, cultural, and deeply ingrained it may be—should ever be substituted for an actual human life.

Accepting something is the first step to changing it, so you’ll have to do better than that.

TimFreeman Jul 9, 2011, 12:35 AM
0 points
in reply to: Raw_Power’s comment on: Dark Arts: Schopenhauer wrote The Book on How To Troll

Please tell me you’ve at least read Methods Of Rationality and Shinji and Warhammer40k.

I read the presently existing part of MoR. I could read Shinji 40K. Why do you think it’s worthwhile? Should I read or watch Neon Genesis Evangelion first?

TimFreeman Jul 8, 2011, 11:13 PM
10 points
on: Dark Arts: Schopenhauer wrote The Book on How To Troll
I have a fear that becoming skilled at bullshitting others will increase my ability to bullshit myself. This is based on my informal observation that the people who bullshit me tend to be a bit confused even when manipulating me isn’t their immediate goal.

However, I do find that being able to authoritatively blame someone else who is using a well-known rhetorical technique for doing that is very useful, and therefore I have found reading “Art of Controversy” to be very useful. The obviously useful skill is to be able to recognize each rhetorical technique and be able to find a suitable retort in real time; the default retort is to name the rhetorical technique.

TimFreeman Jul 6, 2011, 10:03 PM
1 point
on: Behaviorism: Beware Anthropomorphizing Humans

minds are behavior-executors and not utility-maximizers

I think it would be more accurate to say that minds are more accurately and simply modeled as behavior-executors than as utility-maximizers.

There are situations where the most accurate and simple model isn’t the one you want to use. For example, if I’m wanting to cooperate with somebody, one approach is to model them as a utility-maximizer, and then to search for actions that improve everybody’s utility. If I model them as a behavior-executors then I’ll be perceived as manipulative if I don’t get it exactly right.

Specifically, people don’t like to hear an explanation of my behavior of the form “Yes, I reasonably guessed that you would have preferred to get to the grocery store, and I knew that the only grocery store was to the south, but you were driving north so I helped you drive north.” Thus, if I model people as behavior-executors, I have a much more complicated game to play because I have to anticipate what they’ll do when they discover that I helped them to make a mistake.

TimFreeman Jul 6, 2011, 9:07 PM
11 points
on: Topics to discuss CEV
An alternative to CEV is CV, that is, leave out the extrapolation.

You have a bunch of non-extrapolated people now, and I don’t see why we should think their extrapolated desires are morally superior to their present desires. Giving them their extrapolated desires instead of their current desires puts you into conflict with the non-extrapolated version of them, and I’m not sure what worthwhile thing you’re going to get in exchange for that.

Nobody has lived 1000 years yet; maybe extrapolating human desires out to 1000 years gives something that a normal human would say is a symptom of having mental bugs when the brain is used outside the domain for which it was tested, rather than something you’d want an AI to enact. The AI isn’t going to know what’s a bug and what’s a feature.

There’s also a cause-effect cycle with it. My future desires depend on my future experiences, which depend on my interaction with the CEV AI if one is deployed, so the CEV AI’s behavior depends on its estimate of my future desires, which I suppose depends on its estimate of my future experiences, which in turn depends on its estimate of its future behavior. The straightforward way of estimating that has a cycle, and I don’t see why the cycle would converge.

The example in the CEV paper about Fred wanting to murder Steve is better dealt with by acknowledging that Steve wants to live now, IMO, rather than hoping that an extrapolated version of Fred wouldn’t want to commit murder.

ETA: Alternatives include my Respectful AI paper, and Bill Hibbard’s approach. IMO your list of alternatives should include alternatives you disagree with, along with statements about why. Maybe some of the bad solutions have good ideas that are reusable, and maybe pointers to known-bad ideas will save people from writing up another instance of an idea already known to be bad.

IMO, if SIAI really wants the problem to be solved, SIAI should publish a taxonomy of known-bad FAI solutions, along with what’s wrong with them. I am not aware that they have done that. Can anyone point me to such a document?

TimFreeman Jul 5, 2011, 4:31 AM
4 points
on: A summary of Savage’s foundations for probability and utility.
Peter Wakker apparently thinks he found a way to have unbounded utilities and obey most of Savage’s axioms. See Unbounded utility for Savage’s “Foundations of Statistics,” and other models. I’ll say more if and when I understand that paper.

TimFreeman Jun 20, 2011, 6:09 PM
3 points
in reply to: timtyler’s comment on: Science Doesn’t Trust Your Rationality

We can’t use Solomonoff induction—because it is uncomputable.

Generating hypotheses is uncomputable. However, once you have a candidate hypothesis, if it explains the observations you can do a computation to verify that, and you can always measure its complexity. So you’ll never know that you have the best hypothesis, but you can compare hypotheses for quality.

I’d really like to know if there’s anything to be known about the nature of the suboptimal predictions you’ll make if you use suboptimal hypotheses, since we’re pretty much certain to be using suboptimal hypotheses.

TimFreeman Jun 17, 2011, 10:33 PM
0 points
in reply to: timtyler’s comment on: Model Uncertainty, Pascalian Reasoning and Utilitarianism
I agree with jsteinhardt, thanks for the reference.

I agree that the reward functions will vary in complexity. If you do the usual thing in Solomonoff induction, where the plausibility of a reward function decreases exponentially with its size, so far as I can tell you can infer reward fuctions from behavior, if you can infer behavior.

We need to infer a utility function for somebody if we’re going to help them get what they want, since a utility function is the only reasonable description I know of what an agent wants.

TimFreeman Jun 17, 2011, 10:23 PM
0 points
in reply to: jsteinhardt’s comment on: Model Uncertainty, Pascalian Reasoning and Utilitarianism
Surely we can talk about rational agents in other ways that are not so confusing?

Who is sure? If you’re saying that, I hope you are. What do you propose?

Either way, just because something is mathematically proven to exist doesn’t mean that we should have to use it.

I don’t think anybody advocated what you’re arguing against there.

The nearest thing I’m willing to argue for is that one of the following possibilities hold:
- We use something that has been mathematically proven to exist, now.
- We might be speaking nonsense, depending on whether the concepts we’re using can be mathematically proven to make sense in the future.

TimFreeman Jun 16, 2011, 2:35 PM
5 points
in reply to: Will_Newsome’s comment on: Model Uncertainty, Pascalian Reasoning and Utilitarianism

Why are people on Less Wrong still talking about ‘their’ ‘values’ using deviations from a model that assumes they have a ‘utility function’? It’s not enough to explicitly believe and disclaim that this is obviously an incorrect model, at some point you have to actually stop using the model and adopt something else. People are godshatter, they are incoherent, they are inconsistent, they are an abstraction, they are confused about morality, their revealed preferences aren’t their preferences, their revealed preferences aren’t even their revealed preferences, their verbally expressed preferences aren’t even preferences, the beliefs of parts of them about the preferences of other parts of them aren’t their preferences, the beliefs of parts of them aren’t even beliefs, preferences aren’t morality, predisposition isn’t justification, et cetera...

We might make something someday that isn’t godshatter, and we need to practice.

I agree that reforming humans to be rational is hopeless, but it is nevertheless useful to imagine how a rational being would deal with things.

TimFreeman Jun 16, 2011, 2:04 PM
0 points
in reply to: endoself’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

What would you want from an unbounded utility function that you couldn’t get if the math turned out so that only bounded utility functions can be used in a decision procedure?

An actual description of my preferences. I am unsure whether my utility function is actually unbounded but I find it probable that, for example, my utility function is linear in people. I don’t want to rule this out just because that current framework is insufficient for it.

Predicting your preferences requires specifying both the utility function and the framework, so offering a utility function without the framework as an explanation for your preferences does not actually explain them. I actually don’t know if my question was hypothetical or not. Do we have a decision procedure that gives reasonable results for an unbounded utility function?

The phrase “rule this out” seems interesting here. At any given time, you’ll have a set of explanations for your behavior. That doesn’t rule out coming up with better explanations later. Does the best explanation you have for your preferences that works with a known decision theory have bounded utility?

Perhaps I see what’s going on here—people who want unbounded utility are feeling loss when they imagine giving that up that unbounded goodness in order to avoid bugs like the one described in the OP. I, on the other hand, feel loss when people dither over difficult math problems when the actual issues confronting us have nothing to do with difficult math. Specifically, dealing effectively with the default future, in which one or more corporations make AI’s that optimize for something having no connection to the preferences of any individual human.
What links here?
- Wei Dai's comment on A Defense of Naive Metaethics by Will_Sawin (Jun 16, 2011, 5:15 PM; 6 points)

TimFreeman Jun 16, 2011, 1:37 PM
0 points
in reply to: AlephNeil’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility
Thanks for the pointer to a free version of Hajek’s “Waging War on Pascal’s Wager” paper. One of his alternative formulations uses surreal numbers for utilities, much to my surprise.

The main thrust is that either the utility of Heaven isn’t the best possible thing, or it is the best possible thing and a mixed strategy of betting on heaven with probability p and betting on nothing with probability 1-p also gives infinite utility, for positive p. Thus, if Heaven is the best possible thing, Pascal’s Wager doesn’t rule out mixed strategies.

If someone could check my math here—I don’t think surreal numbers let you assign a utility to the St. Petersburg paradox. The expected utility received at each step is 1, so the total utility is 1 + 1 + 1 + … . Suppose that sum is X. Then X + 1 = X. This is not true for any surreal number, right?

TimFreeman Jun 12, 2011, 3:34 AM
0 points
in reply to: CarlShulman’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

See Alan Hajek’s classic article “Waging War on Pascal’s Wager.”

That article is paywalled. It was published in 2003. Hajek’s entry about Pascal’s Wager in the Stanford Encylopedia of Philosophy is free and was substantively revised (hopefully by Hajek) in 2008, so there’s a good chance the latter contains all the good ideas in the former and is easier to get to. The latter does mention the idea that utilities should be bounded, and many other things potentially wrong with Pascal’s wager. There’s no neat list of four items that looks like an obvious match to the title of the paywalled article.
What links here?
- TimFreeman's comment on St. Petersburg Mugging Implies You Have Bounded Utility by TimFreeman (Jun 7, 2011, 8:40 PM; 0 points)

TimFreeman Jun 9, 2011, 4:30 PM
1 point
in reply to: endoself’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility
Yes, the utility function is not up for grabs, but introspection doesn’t tell you what it is either. In particular, the statement “endoself acts approximately consistently with utility function U” is an empirical statement for any given U (and any particular notion of “approximately”, but let’s skip that part for now). I believe I have provided fine arguments that you are not acting approximately consistently with an unbounded utility function, and that you will never be able to do so. If those arguments are valid, and you say you have an unbounded utility function, then you are wrong.

If those arguments are valid, and you say you want to have an unbounded utility function, then you’re wanting something impossible because you falsely believe it to be possible. The best I could do in that case if I were helping you would be to give you what you would want if you had true beliefs. I don’t know what that would be. What would you want from an unbounded utility function that you couldn’t get if the math turned out so that only bounded utility functions can be used in a decision procedure?

you specified that small changes to likely gods are unlikely gods, there is a distinction in that there are not a metaphorical million of them

There are many paths by which small actions taken today might in unlikely ways influence the details of how a likely god is built. If those paths have infinite utility, you have to analyze them to decide what to do.

TimFreeman Jun 9, 2011, 4:13 PM
0 points
in reply to: Eugine_Nier’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

So how about avoiding your version by saying that all the terms in my utility function are bounded except for the ones that scale linearly with the number of people?

That seems hackish, but given a mathematical definition of “person” it might be implementable. I don’t know what that definition would be. Given that we don’t have a clear definition for “person”, Hanson’s proposal (and anthropic arguments in general) seem like bunk to me.

TimFreeman Jun 8, 2011, 2:50 AM
−2 points
in reply to: endoself’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

Even then, low-probability interactions between eating an apple and the nature of the likely god seem likely to lead to bizarre decision processes about apple-eating, unless you have bounded utilities.

What causes you to find it so unlikely that our desires could work this way?

Pay attention next time you eat something. Do you look at the food and eat what you like or what you think will improve your health, or do you try to prioritize eating the food against sending me money because I might be a god, and against giving all of the other unlikely gods what they might want?

We are human and cannot really do that. With unbounded utilities, there are an absurdly large number of possible ways that an ordinary action can have very low-probability influence on a wide variety of very high-utility things, and you have to take them all into account and balance them properly to do the right thing. If an AI is doing that, I have no confidence at all that it will weigh these things the way I would like, especially given that it’s not likely to search all of the space. Someone who thinks about a million unlikely gods to decide whether to eat an apple is broken. In practice, they won’t be able to do that, and their decision about whether to eat the apple will be driven by whatever unlikely gods have been brought to their attention in the last minute. (As I said before, an improbable minor change to a likely god is an unlikely god, for the purposes of this discussion.)

If utilities are bounded, then the number of alternatives you have to look at doesn’t grow so pathologically large, and you look at the apple to decide whether to eat the apple. The unlikely gods don’t enter into it because you don’t imagine that they can make enough of a difference to outweigh their unlikeliness.

TimFreeman Jun 8, 2011, 1:55 AM
1 point
in reply to: endoself’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

I really care whether they can pay attention to an apple when they’re hungry to decide whether to eat the apple, as opposed to considering obscure consequences of how various possible unlikely gods might react to the eating of the apple.

Personal survival makes it a lot easier to please unlikely gods, so eating the apple is preferred.

Eating an apple was meant to be an example of a trivial thing. Inflating it to personal survival misses the point. Eating an apple should be connected to your own personal values concerning hunger and apples, and there should be a way to make a decision about eating an apple or not eating it and being slightly hungry based on your personal values about apples and hunger. If we have to think about unlikely gods to decide whether to eat an apple, something is broken.

For more general situations, some paths to infinity are much more probable than others. For example, perhaps we can build a god [SIAI pointer].

That’s a likely god, not an unlikely god, so it’s a little bit different. Even then, low-probability interactions between eating an apple and the nature of the likely god seem likely to lead to bizarre decision processes about apple-eating, unless you have bounded utilities.

TimFreeman Jun 7, 2011, 10:43 PM
0 points
in reply to: benelliott’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

Surely if you were God you would be able to instantly work out BB(n) for any n, which would make you uncomputable, which would indeed mean the Solomonoff prior assigns you being God a probability of zero.

I agree that the Solomonoff prior isn’t going to give positive probability to me having any sort of halting oracle. Hmm, I’m not sure whether inferring someone’s utility function is computable. I suppose that inferring the utility function for a brain of fixed complexity when arbitrarily large (but still finite) computational capacity can be brought to bear could give an arbitrarily close approximation, so the OP could be revised to fix that. It presently doesn’t seem worth the effort though—the added verbage would obscure the main point without adding anything obviously useful.

TimFreeman Jun 7, 2011, 8:47 PM
1 point
in reply to: XiXiDu’s comment on: St. Petersburg Mugging Implies You Have Bounded Utility

This problem is the reason for most of the headache that LW is causing me and I appreciate any attention it receives.

Me too. Would vote you up twice if I could.

Even mathematicians like John Baez are troubled by the unbounded maximization of expected utility.

I don’t think he mentioned “unbounded” in the post you’re citing. He talked about risk aversion, and that can be encoded by changing the utility function.