Utility Functions

TagLast edit: 22 Sep 2020 15:35 UTC by abramdemski

A utility function assigns numerical values (“utilities”) to outcomes, in such a way that outcomes with higher utilities are always preferred to outcomes with lower utilities.

See also: Complexity of Value, Decision Theory, Game Theory, Orthogonality Thesis, Utilitarianism, Preference, Utility, VNM Theorem

Utility Functions do not work very well in practice for individual humans. Human drives are not coherent nor is there any reason to think they would be (Thou Art Godshatter), and even people with a strong interest in the concept have trouble working out what their utility function actually is even slightly (Post Your Utility Function). Furthermore, humans appear to calculate utility and disutility separately—adding one to the other does not predict their behavior accurately. This makes humans highly exploitable.

pjeby posits humans’ difficulty in understanding their own utility functions as the root of akrasia.

However, utility functions can be a useful model for dealing with humans in groups, e.g. in economics.

The VNM Theorem tag is likely to be a strict subtag of the Utility Functions tag, because the VNM theorem establishes when preferences can be represented by a utility function, but a post discussing utility functions may or may not discuss the VNM theorem/​axioms.

Co­her­ent de­ci­sions im­ply con­sis­tent utilities

Eliezer Yudkowsky12 May 2019 21:33 UTC
124 points
80 comments26 min readLW link3 nominations3 reviews

An Ortho­dox Case Against Utility Functions

abramdemski7 Apr 2020 19:18 UTC
116 points
50 comments8 min readLW link

Co­her­ence ar­gu­ments do not im­ply goal-di­rected behavior

rohinmshah3 Dec 2018 3:26 UTC
83 points
65 comments7 min readLW link

Utility ≠ Reward

vlad_m5 Sep 2019 17:28 UTC
97 points
24 comments12 min readLW link2 nominations2 reviews

How eas­ily can we sep­a­rate a friendly AI in de­sign space from one which would bring about a hy­per­ex­is­ten­tial catas­tro­phe?

Anirandis10 Sep 2020 0:40 UTC
18 points
20 comments2 min readLW link

The Hu­man’s Hid­den Utility Func­tion (Maybe)

lukeprog23 Jan 2012 19:39 UTC
62 points
88 comments3 min readLW link

Re­search Agenda v0.9: Syn­the­sis­ing a hu­man’s prefer­ences into a util­ity function

Stuart_Armstrong17 Jun 2019 17:46 UTC
63 points
20 comments33 min readLW link

Against util­ity functions

Qiaochu_Yuan19 Jun 2014 5:56 UTC
65 points
87 comments1 min readLW link

Us­ing ex­pected util­ity for Good(hart)

Stuart_Armstrong27 Aug 2018 3:32 UTC
42 points
5 comments4 min readLW link

The VNM in­de­pen­dence ax­iom ig­nores the value of information

kilobug2 Mar 2013 14:36 UTC
15 points
48 comments1 min readLW link

Time and Effort Discounting

Scott Alexander7 Jul 2011 23:48 UTC
62 points
32 comments4 min readLW link

The Fun­da­men­tal The­o­rem of As­set Pric­ing: Miss­ing Link of the Dutch Book Arguments

johnswentworth1 Jun 2019 20:34 UTC
42 points
5 comments3 min readLW link

Deon­tol­ogy for Consequentialists

Alicorn30 Jan 2010 17:58 UTC
52 points
255 comments6 min readLW link

When do util­ity func­tions con­strain?

Hoagy23 Aug 2019 17:19 UTC
27 points
5 comments7 min readLW link

[link] Choose your (prefer­ence) util­i­tar­i­anism care­fully – part 1

Kaj_Sotala25 Jun 2015 12:06 UTC
21 points
6 comments2 min readLW link

To cap­ture anti-death in­tu­itions, in­clude mem­ory in utilitarianism

Kaj_Sotala15 Jan 2014 6:27 UTC
12 points
34 comments3 min readLW link

Choos­ing the Zero Point

orthonormal6 Apr 2020 23:44 UTC
145 points
20 comments3 min readLW link

The Allais Paradox

Eliezer Yudkowsky19 Jan 2008 3:05 UTC
50 points
140 comments3 min readLW link

Stable Poin­t­ers to Value III: Re­cur­sive Quantilization

abramdemski21 Jul 2018 8:06 UTC
19 points
4 comments4 min readLW link

Pin­point­ing Utility

[deleted]1 Feb 2013 3:58 UTC
93 points
156 comments13 min readLW link

The Iso­la­tion As­sump­tion of Ex­pected Utility Maximization

Pedro Oliboni6 Aug 2020 4:05 UTC
7 points
1 comment5 min readLW link

AI Align­ment 2018-19 Review

rohinmshah28 Jan 2020 2:19 UTC
121 points
6 comments35 min readLW link

Per­son-mo­ment af­fect­ing views

KatjaGrace7 Mar 2018 2:30 UTC
17 points
8 comments5 min readLW link

Com­par­ing Utilities

abramdemski14 Sep 2020 20:56 UTC
61 points
31 comments17 min readLW link

In­terthe­o­retic util­ity comparison

Stuart_Armstrong3 Jul 2018 13:44 UTC
23 points
11 comments6 min readLW link

If you don’t know the name of the game, just tell me what I mean to you

Stuart_Armstrong26 Oct 2010 13:43 UTC
16 points
26 comments5 min readLW link

Com­pu­ta­tional effi­ciency rea­sons not to model VNM-ra­tio­nal prefer­ence re­la­tions with util­ity functions

AlexMennen25 Jul 2018 2:11 UTC
16 points
5 comments3 min readLW link

An At­tempt at Prefer­ence Uncer­tainty Us­ing VNM

[deleted]16 Jul 2013 5:20 UTC
15 points
33 comments6 min readLW link

Why Subagents?

johnswentworth1 Aug 2019 22:17 UTC
126 points
34 comments7 min readLW link2 nominations1 review

Co­her­ence ar­gu­ments im­ply a force for goal-di­rected behavior

KatjaGrace26 Mar 2021 16:10 UTC
86 points
20 comments14 min readLW link

I’m no longer sure that I buy dutch book ar­gu­ments and this makes me skep­ti­cal of the “util­ity func­tion” abstraction

Eli Tyre22 Jun 2021 3:53 UTC
42 points
28 comments4 min readLW link

[Question] How do bounded util­ity func­tions work if you are un­cer­tain how close to the bound your util­ity is?

Ghatanathoah6 Oct 2021 21:31 UTC
6 points
17 comments2 min readLW link

Hu­mans are util­ity monsters

PhilGoetz16 Aug 2013 21:05 UTC
103 points
217 comments2 min readLW link

[Question] “Do Noth­ing” util­ity func­tion, 3½ years later?

niplav20 Jul 2020 11:09 UTC
5 points
3 comments1 min readLW link

De­grees of Freedom

sarahconstantin2 Apr 2019 21:10 UTC
102 points
31 comments11 min readLW link

Against the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Penalty

AlexMennen14 Dec 2017 18:38 UTC
39 points
47 comments11 min readLW link

Ter­mi­nal Values and In­stru­men­tal Values

Eliezer Yudkowsky15 Nov 2007 7:56 UTC
81 points
42 comments10 min readLW link

Three ways that “Suffi­ciently op­ti­mized agents ap­pear co­her­ent” can be false

Wei_Dai5 Mar 2019 21:52 UTC
63 points
3 comments3 min readLW link

The ge­nie knows, but doesn’t care

Rob Bensinger6 Sep 2013 6:42 UTC
88 points
519 comments8 min readLW link

Buri­dan’s ass in co­or­di­na­tion games

jessicata16 Jul 2018 2:51 UTC
52 points
26 comments10 min readLW link

Pas­cal’s Mug­ging: Tiny Prob­a­bil­ities of Vast Utilities

Eliezer Yudkowsky19 Oct 2007 23:37 UTC
77 points
353 comments4 min readLW link

Pas­cal’s Mug­gle: In­finites­i­mal Pri­ors and Strong Evidence

Eliezer Yudkowsky8 May 2013 0:43 UTC
67 points
406 comments26 min readLW link

We Don’t Have a Utility Function

[deleted]2 Apr 2013 3:49 UTC
70 points
124 comments4 min readLW link

Prob­a­bil­ity is Real, and Value is Complex

abramdemski20 Jul 2018 5:24 UTC
59 points
18 comments6 min readLW link

The Lifes­pan Dilemma

Eliezer Yudkowsky10 Sep 2009 18:45 UTC
52 points
220 comments7 min readLW link

When to use quantilization

RyanCarey5 Feb 2019 17:17 UTC
53 points
5 comments4 min readLW link

Big Ad­vance in In­finite Ethics

bwest28 Nov 2017 15:10 UTC
27 points
14 comments5 min readLW link

Fake Utility Functions

Eliezer Yudkowsky6 Dec 2007 16:55 UTC
52 points
62 comments4 min readLW link

More on the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Prior

AlexMennen26 Feb 2018 23:53 UTC
16 points
4 comments9 min readLW link

Ex­pected util­ity, un­los­ing agents, and Pas­cal’s mugging

Stuart_Armstrong28 Jul 2014 18:05 UTC
32 points
54 comments5 min readLW link

Is risk aver­sion re­ally ir­ra­tional ?

kilobug31 Jan 2012 20:34 UTC
50 points
65 comments9 min readLW link

Co­her­ent be­havi­our in the real world is an in­co­her­ent concept

Richard_Ngo11 Feb 2019 17:00 UTC
48 points
17 comments9 min readLW link

A sum­mary of Sav­age’s foun­da­tions for prob­a­bil­ity and util­ity.

Sniffnoy22 May 2011 19:56 UTC
68 points
91 comments13 min readLW link

Log­a­r­ithms and To­tal Utilitarianism

pvs9 Aug 2018 8:49 UTC
37 points
31 comments4 min readLW link

Ten­den­cies in re­flec­tive equilibrium

Scott Alexander20 Jul 2011 10:38 UTC
52 points
70 comments4 min readLW link

Un­der­ap­pre­ci­ated points about util­ity func­tions (of both sorts)

Sniffnoy4 Jan 2020 7:27 UTC
34 points
61 comments15 min readLW link

Zut Allais!

Eliezer Yudkowsky20 Jan 2008 3:18 UTC
40 points
49 comments6 min readLW link

Against Dis­count Rates

Eliezer Yudkowsky21 Jan 2008 10:00 UTC
32 points
81 comments2 min readLW link

Why you must max­i­mize ex­pected utility

Benya13 Dec 2012 1:11 UTC
47 points
77 comments21 min readLW link

Only hu­mans can have hu­man values

PhilGoetz26 Apr 2010 18:57 UTC
51 points
161 comments17 min readLW link

Post Your Utility Function

taw4 Jun 2009 5:05 UTC
32 points
280 comments1 min readLW link

Ap­ply­ing util­ity func­tions to hu­mans con­sid­ered harmful

Kaj_Sotala3 Feb 2010 19:22 UTC
35 points
116 comments5 min readLW link

What re­sources have in­creas­ing marginal util­ity?

Qiaochu_Yuan14 Jun 2014 3:43 UTC
59 points
63 comments1 min readLW link

Bayesian Utility: Rep­re­sent­ing Prefer­ence by Prob­a­bil­ity Measures

Vladimir_Nesov27 Jul 2009 14:28 UTC
38 points
36 comments2 min readLW link

The Do­main of Your Utility Function

Peter_de_Blanc23 Jun 2009 4:58 UTC
40 points
99 comments2 min readLW link

Sim­plified prefer­ences needed; sim­plified prefer­ences sufficient

Stuart_Armstrong5 Mar 2019 19:39 UTC
29 points
6 comments3 min readLW link

Utility ver­sus Re­ward func­tion: par­tial equivalence

Stuart_Armstrong13 Apr 2018 14:58 UTC
8 points
5 comments5 min readLW link

The Prefer­ence Utili­tar­ian’s Time In­con­sis­tency Problem

Wei_Dai15 Jan 2010 0:26 UTC
34 points
107 comments1 min readLW link

Com­plex Be­hav­ior from Sim­ple (Sub)Agents

moridinamael10 May 2019 21:44 UTC
107 points
13 comments9 min readLW link2 nominations1 review

“Solv­ing” self­ish­ness for UDT

Stuart_Armstrong27 Oct 2014 17:51 UTC
38 points
50 comments8 min readLW link

Why Univer­sal Com­pa­ra­bil­ity of Utility?

AK13 May 2018 0:10 UTC
8 points
16 comments1 min readLW link

What we talk about when we talk about max­imis­ing utility

Richard_Ngo24 Feb 2018 22:33 UTC
14 points
18 comments4 min readLW link

VNM ex­pected util­ity the­ory: uses, abuses, and interpretation

Academian17 Apr 2010 20:23 UTC
35 points
51 comments10 min readLW link

Risk aver­sion vs. con­cave util­ity function

dvasya31 Jan 2012 6:25 UTC
3 points
35 comments3 min readLW link

Univer­sal agents and util­ity functions

Anja14 Nov 2012 4:05 UTC
43 points
38 comments6 min readLW link

Harsanyi’s So­cial Ag­gre­ga­tion The­o­rem and what it means for CEV

AlexMennen5 Jan 2013 21:38 UTC
37 points
88 comments4 min readLW link

Ver­ify­ing vNM-ra­tio­nal­ity re­quires an ontology

jeyoor13 Mar 2019 0:03 UTC
24 points
5 comments1 min readLW link

Are pre-speci­fied util­ity func­tions about the real world pos­si­ble in prin­ci­ple?

mlogan11 Jul 2018 18:46 UTC
24 points
7 comments4 min readLW link

Sublimity vs. Youtube

Alicorn18 Mar 2011 5:33 UTC
31 points
61 comments1 min readLW link

Why the be­liefs/​val­ues di­chotomy?

Wei_Dai20 Oct 2009 16:35 UTC
28 points
156 comments2 min readLW link

Knigh­tian Uncer­tainty and Am­bi­guity Aver­sion: Motivation

So8res21 Jul 2014 20:32 UTC
36 points
44 comments13 min readLW link

Con­cep­tual prob­lems with util­ity functions

Dacyn11 Jul 2018 1:29 UTC
22 points
12 comments2 min readLW link

Allais Malaise

Eliezer Yudkowsky21 Jan 2008 0:40 UTC
31 points
38 comments2 min readLW link

I’m con­fused. Could some­one help?

CronoDAS23 Mar 2009 5:26 UTC
1 point
13 comments1 min readLW link

On dol­lars, util­ity, and crack cocaine

PhilGoetz4 Apr 2009 0:00 UTC
16 points
100 comments2 min readLW link

Real-world ex­am­ples of money-pump­ing?

sixes_and_sevens25 Apr 2013 13:49 UTC
28 points
97 comments1 min readLW link

How Not to be Stupid: Adorable Maybes

Psy-Kosh29 Apr 2009 19:15 UTC
1 point
55 comments3 min readLW link

Ex­pected util­ity with­out the in­de­pen­dence axiom

Stuart_Armstrong28 Oct 2009 14:40 UTC
20 points
68 comments4 min readLW link

Allais Hack—Trans­form Your De­ci­sions!

MBlume3 May 2009 22:37 UTC
22 points
19 comments2 min readLW link

How Not to be Stupid: Brew­ing a Nice Cup of Utilitea

Psy-Kosh9 May 2009 8:14 UTC
2 points
18 comments6 min readLW link

Want­ing to Want

Alicorn16 May 2009 3:08 UTC
22 points
199 comments2 min readLW link

Ex­pected fu­til­ity for humans

Roko9 Jun 2009 12:04 UTC
13 points
53 comments3 min readLW link

If it looks like util­ity max­i­mizer and quacks like util­ity max­i­mizer...

taw11 Jun 2009 18:34 UTC
20 points
24 comments2 min readLW link

Utility Max­i­miza­tion = De­scrip­tion Length Minimization

johnswentworth18 Feb 2021 18:04 UTC
148 points
35 comments5 min readLW link

A fun­gi­bil­ity theorem

Nisan12 Jan 2013 9:27 UTC
35 points
66 comments6 min readLW link

Chas­ing Infinities

Michael Bateman16 Aug 2021 1:19 UTC
2 points
1 comment9 min readLW link

The Dou­bling Box

Mestroyer6 Aug 2012 5:50 UTC
22 points
83 comments3 min readLW link
No comments.