RSS

Utility Functions

A util­ity func­tion as­signs nu­mer­i­cal val­ues (“util­ities”) to out­comes, in such a way that out­comes with higher util­ities are always preferred to out­comes with lower util­ities.

See also: Com­plex­ity of Value, De­ci­sion The­ory, Game The­ory, Orthog­o­nal­ity Th­e­sis, Utili­tar­i­anism, Prefer­ence, Utility

Utility Func­tions do not work very well in prac­tice for in­di­vi­d­ual hu­mans. Hu­man drives are not co­her­ent nor is there any rea­son to think they would be (Thou Art God­shat­ter), and even peo­ple with a strong in­ter­est in the con­cept have trou­ble work­ing out what their util­ity func­tion ac­tu­ally is even slightly (Post Your Utility Func­tion). Fur­ther­more, hu­mans ap­pear to calcu­late util­ity and di­su­til­ity sep­a­rately—adding one to the other does not pre­dict their be­hav­ior ac­cu­rately. This makes hu­mans highly ex­ploitable.

pjeby posits hu­mans’ difficulty in un­der­stand­ing their own util­ity func­tions as the root of akra­sia.

How­ever, util­ity func­tions can be a use­ful model for deal­ing with hu­mans in groups, e.g. in eco­nomics.

Co­her­ent de­ci­sions im­ply con­sis­tent utilities

Eliezer Yudkowsky
12 May 2019 21:33 UTC
122 points
56 comments26 min readLW link

An Ortho­dox Case Against Utility Functions

abramdemski
7 Apr 2020 19:18 UTC
118 points
49 comments8 min readLW link

Co­her­ence ar­gu­ments do not im­ply goal-di­rected behavior

rohinmshah
3 Dec 2018 3:26 UTC
79 points
65 comments7 min readLW link2 nominations3 reviews

Utility ≠ Reward

vlad_m
5 Sep 2019 17:28 UTC
79 points
18 comments12 min readLW link

How eas­ily can we sep­a­rate a friendly AI in de­sign space from one which would bring about a hy­per­ex­is­ten­tial catas­tro­phe?

Anirandis
10 Sep 2020 0:40 UTC
18 points
20 comments2 min readLW link

The Hu­man’s Hid­den Utility Func­tion (Maybe)

lukeprog
23 Jan 2012 19:39 UTC
47 points
88 comments3 min readLW link

The Allais Paradox

Eliezer Yudkowsky
19 Jan 2008 3:05 UTC
39 points
138 comments3 min readLW link

Pin­point­ing Utility

[deleted]
1 Feb 2013 3:58 UTC
57 points
156 comments13 min readLW link

Re­search Agenda v0.9: Syn­the­sis­ing a hu­man’s prefer­ences into a util­ity function

Stuart_Armstrong
17 Jun 2019 17:46 UTC
64 points
20 comments33 min readLW link

Against util­ity functions

Qiaochu_Yuan
19 Jun 2014 5:56 UTC
43 points
87 comments1 min readLW link

Us­ing ex­pected util­ity for Good(hart)

Stuart_Armstrong
27 Aug 2018 3:32 UTC
44 points
5 comments4 min readLW link

The VNM in­de­pen­dence ax­iom ig­nores the value of information

kilobug
2 Mar 2013 14:36 UTC
12 points
48 comments1 min readLW link

Time and Effort Discounting

Scott Alexander
7 Jul 2011 23:48 UTC
47 points
32 comments4 min readLW link

The Fun­da­men­tal The­o­rem of As­set Pric­ing: Miss­ing Link of the Dutch Book Arguments

johnswentworth
1 Jun 2019 20:34 UTC
43 points
5 comments3 min readLW link

Deon­tol­ogy for Consequentialists

Alicorn
30 Jan 2010 17:58 UTC
47 points
255 comments6 min readLW link

When do util­ity func­tions con­strain?

Hoagy
23 Aug 2019 17:19 UTC
18 points
5 comments7 min readLW link

[link] Choose your (prefer­ence) util­i­tar­i­anism care­fully – part 1

Kaj_Sotala
25 Jun 2015 12:06 UTC
15 points
6 comments2 min readLW link

To cap­ture anti-death in­tu­itions, in­clude mem­ory in utilitarianism

Kaj_Sotala
15 Jan 2014 6:27 UTC
8 points
34 comments3 min readLW link

Choos­ing the Zero Point

orthonormal
6 Apr 2020 23:44 UTC
136 points
20 comments3 min readLW link

Stable Poin­t­ers to Value III: Re­cur­sive Quantilization

abramdemski
21 Jul 2018 8:06 UTC
20 points
4 comments4 min readLW link

The Iso­la­tion As­sump­tion of Ex­pected Utility Maximization

Pedro Oliboni
6 Aug 2020 4:05 UTC
7 points
1 comment5 min readLW link

AI Align­ment 2018-19 Review

rohinmshah
28 Jan 2020 2:19 UTC
143 points
6 comments35 min readLW link

Per­son-mo­ment af­fect­ing views

KatjaGrace
7 Mar 2018 2:30 UTC
42 points
8 comments5 min readLW link
(meteuphoric.wordpress.com)

Com­par­ing Utilities

abramdemski
14 Sep 2020 20:56 UTC
44 points
31 comments17 min readLW link

In­terthe­o­retic util­ity comparison

Stuart_Armstrong
3 Jul 2018 13:44 UTC
23 points
11 comments6 min readLW link

If you don’t know the name of the game, just tell me what I mean to you

Stuart_Armstrong
26 Oct 2010 13:43 UTC
13 points
26 comments5 min readLW link

Hu­mans are util­ity monsters

PhilGoetz
16 Aug 2013 21:05 UTC
71 points
216 comments2 min readLW link

[Question] “Do Noth­ing” util­ity func­tion, 3½ years later?

niplav
20 Jul 2020 11:09 UTC
5 points
3 comments1 min readLW link

De­grees of Freedom

sarahconstantin
2 Apr 2019 21:10 UTC
112 points
31 comments11 min readLW link
(srconstantin.wordpress.com)

Against the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Penalty

AlexMennen
14 Dec 2017 18:38 UTC
73 points
47 comments11 min readLW link

Ter­mi­nal Values and In­stru­men­tal Values

Eliezer Yudkowsky
15 Nov 2007 7:56 UTC
70 points
41 comments10 min readLW link

Three ways that “Suffi­ciently op­ti­mized agents ap­pear co­her­ent” can be false

Wei_Dai
5 Mar 2019 21:52 UTC
69 points
3 comments3 min readLW link

The ge­nie knows, but doesn’t care

Rob Bensinger
6 Sep 2013 6:42 UTC
59 points
518 comments8 min readLW link

Buri­dan’s ass in co­or­di­na­tion games

jessicata
16 Jul 2018 2:51 UTC
55 points
26 comments10 min readLW link

Pas­cal’s Mug­ging: Tiny Prob­a­bil­ities of Vast Utilities

Eliezer Yudkowsky
19 Oct 2007 23:37 UTC
64 points
346 comments4 min readLW link

Pas­cal’s Mug­gle: In­finites­i­mal Pri­ors and Strong Evidence

Eliezer Yudkowsky
8 May 2013 0:43 UTC
48 points
406 comments26 min readLW link

We Don’t Have a Utility Function

[deleted]
2 Apr 2013 3:49 UTC
47 points
124 comments4 min readLW link

Prob­a­bil­ity is Real, and Value is Complex

abramdemski
20 Jul 2018 5:24 UTC
51 points
18 comments6 min readLW link

The Lifes­pan Dilemma

Eliezer Yudkowsky
10 Sep 2009 18:45 UTC
41 points
219 comments7 min readLW link

When to use quantilization

RyanCarey
5 Feb 2019 17:17 UTC
61 points
5 comments4 min readLW link

Big Ad­vance in In­finite Ethics

bwest
28 Nov 2017 15:10 UTC
55 points
14 comments5 min readLW link

Fake Utility Functions

Eliezer Yudkowsky
6 Dec 2007 16:55 UTC
40 points
62 comments4 min readLW link

More on the Lin­ear Utility Hy­poth­e­sis and the Lev­er­age Prior

AlexMennen
26 Feb 2018 23:53 UTC
37 points
4 comments9 min readLW link

Ex­pected util­ity, un­los­ing agents, and Pas­cal’s mugging

Stuart_Armstrong
28 Jul 2014 18:05 UTC
19 points
54 comments5 min readLW link

Is risk aver­sion re­ally ir­ra­tional ?

kilobug
31 Jan 2012 20:34 UTC
41 points
65 comments9 min readLW link

Co­her­ent be­havi­our in the real world is an in­co­her­ent concept

Richard_Ngo
11 Feb 2019 17:00 UTC
40 points
17 comments9 min readLW link

A sum­mary of Sav­age’s foun­da­tions for prob­a­bil­ity and util­ity.

Sniffnoy
22 May 2011 19:56 UTC
45 points
91 comments13 min readLW link

Log­a­r­ithms and To­tal Utilitarianism

pvs
9 Aug 2018 8:49 UTC
39 points
31 comments4 min readLW link

Ten­den­cies in re­flec­tive equilibrium

Scott Alexander
20 Jul 2011 10:38 UTC
41 points
70 comments4 min readLW link

Un­der­ap­pre­ci­ated points about util­ity func­tions (of both sorts)

Sniffnoy
4 Jan 2020 7:27 UTC
27 points
61 comments15 min readLW link

Zut Allais!

Eliezer Yudkowsky
20 Jan 2008 3:18 UTC
28 points
48 comments6 min readLW link

Against Dis­count Rates

Eliezer Yudkowsky
21 Jan 2008 10:00 UTC
26 points
81 comments2 min readLW link

Why you must max­i­mize ex­pected utility

Benya
13 Dec 2012 1:11 UTC
25 points
76 comments21 min readLW link

Only hu­mans can have hu­man values

PhilGoetz
26 Apr 2010 18:57 UTC
37 points
161 comments17 min readLW link

Post Your Utility Function

taw
4 Jun 2009 5:05 UTC
30 points
280 comments1 min readLW link

Ap­ply­ing util­ity func­tions to hu­mans con­sid­ered harmful

Kaj_Sotala
3 Feb 2010 19:22 UTC
33 points
116 comments5 min readLW link

What re­sources have in­creas­ing marginal util­ity?

Qiaochu_Yuan
14 Jun 2014 3:43 UTC
37 points
63 comments1 min readLW link

Bayesian Utility: Rep­re­sent­ing Prefer­ence by Prob­a­bil­ity Measures

Vladimir_Nesov
27 Jul 2009 14:28 UTC
33 points
36 comments2 min readLW link

The Do­main of Your Utility Function

Peter_de_Blanc
23 Jun 2009 4:58 UTC
32 points
99 comments2 min readLW link

Sim­plified prefer­ences needed; sim­plified prefer­ences sufficient

Stuart_Armstrong
5 Mar 2019 19:39 UTC
31 points
6 comments3 min readLW link

Utility ver­sus Re­ward func­tion: par­tial equivalence

Stuart_Armstrong
13 Apr 2018 14:58 UTC
25 points
5 comments5 min readLW link

The Prefer­ence Utili­tar­ian’s Time In­con­sis­tency Problem

Wei_Dai
15 Jan 2010 0:26 UTC
27 points
107 comments1 min readLW link

Com­plex Be­hav­ior from Sim­ple (Sub)Agents

moridinamael
10 May 2019 21:44 UTC
103 points
9 comments9 min readLW link

“Solv­ing” self­ish­ness for UDT

Stuart_Armstrong
27 Oct 2014 17:51 UTC
27 points
50 comments8 min readLW link

Why Univer­sal Com­pa­ra­bil­ity of Utility?

AK
13 May 2018 0:10 UTC
27 points
16 comments1 min readLW link

What we talk about when we talk about max­imis­ing utility

Richard_Ngo
24 Feb 2018 22:33 UTC
27 points
18 comments4 min readLW link

VNM ex­pected util­ity the­ory: uses, abuses, and interpretation

Academian
17 Apr 2010 20:23 UTC
25 points
51 comments10 min readLW link

Risk aver­sion vs. con­cave util­ity function

dvasya
31 Jan 2012 6:25 UTC
3 points
35 comments3 min readLW link

Univer­sal agents and util­ity functions

Anja
14 Nov 2012 4:05 UTC
29 points
38 comments6 min readLW link

Harsanyi’s So­cial Ag­gre­ga­tion The­o­rem and what it means for CEV

AlexMennen
5 Jan 2013 21:38 UTC
26 points
88 comments4 min readLW link

Ver­ify­ing vNM-ra­tio­nal­ity re­quires an ontology

jeyoor
13 Mar 2019 0:03 UTC
26 points
5 comments1 min readLW link

Are pre-speci­fied util­ity func­tions about the real world pos­si­ble in prin­ci­ple?

mlogan
11 Jul 2018 18:46 UTC
26 points
7 comments4 min readLW link

Sublimity vs. Youtube

Alicorn
18 Mar 2011 5:33 UTC
23 points
61 comments1 min readLW link