# Bayes’ The­o­rem /​ Bayesianism

Bayes’ The­o­rem (also known as Bayes’ Law) is a law of prob­a­bil­ity that de­scribes the proper way to in­cor­po­rate new ev­i­dence into prior prob­a­bil­ities to form an up­dated prob­a­bil­ity es­ti­mate. It is com­monly re­garded as the foun­da­tion of con­sis­tent ra­tio­nal rea­son­ing un­der un­cer­tainty. Bayes The­o­rem is named af­ter Rev­erend Thomas Bayes who proved the the­o­rem in 1763. Bayesi­anism is the broader philos­o­phy in­spired by the the­o­rem. The core claim be­hind all va­ri­eties of Bayesi­anism is that prob­a­bil­ities are sub­jec­tive de­grees of be­lief—of­ten op­er­a­tional­ized as will­ing­ness to bet.

This stands in con­trast to other in­ter­pre­ta­tions of prob­a­bil­ity, which at­tempt greater ob­jec­tivity. The fre­quen­tist in­ter­pre­ta­tion of prob­a­bil­ity has a fo­cus on re­peat­able ex­per­i­ments; prob­a­bil­ities are the limit­ing fre­quency of an event if you performed the ex­per­i­ment an in­finite num­ber of times.

Another con­tender is the propen­sity in­ter­pre­ta­tion, which grounds prob­a­bil­ity in the propen­sity for things to hap­pen. A perfectly bal­anced 6-sided die would have a 16 propen­sity to land on each side. A propen­sity the­o­rist sees this as a ba­sic fact about dice not de­rived from in­finite se­quences of ex­per­i­ments or sub­jec­tive view­points.

Note how both of these al­ter­na­tive in­ter­pre­ta­tions ground the mean­ing of prob­a­bil­ity in an ex­ter­nal ob­jec­tive fact which can­not be di­rectly ac­cessed.

As a con­se­quence of the sub­jec­tive in­ter­pre­ta­tion of prob­a­bil­ity the­ory, Bayesi­ans are more in­clined to ap­ply Bayes’ The­o­rem in prac­ti­cal statis­ti­cal in­fer­ence. The pri­mary ex­am­ple of this is statis­ti­cal hy­poth­e­sis test­ing. Fre­quen­tists take the ap­pli­ca­tion of Bayes’ The­o­rem to be in­ap­pro­pri­ate, be­cause “the prob­a­bil­ity of a hy­poth­e­sis” is mean­ingless: a hy­poth­e­sis is ei­ther true or false; you can­not define a re­peated ex­per­i­ment in which it is some­times true and some­times false, so you can­not as­sign it an in­ter­me­di­ate prob­a­bil­ity.

Bayes’ the­o­rem com­monly takes the form:

where A is the propo­si­tion of in­ter­est, B is the ob­served ev­i­dence, P(A) and P(B) are prior prob­a­bil­ities, and P(A|B) is the pos­te­rior prob­a­bil­ity of A.

With the pos­te­rior odds, the prior odds and the like­li­hood ra­tio writ­ten ex­plic­itly, the the­o­rem reads:

# An In­tu­itive Ex­pla­na­tion of Bayes’s Theorem

1 Jan 2003 20:00 UTC
28 points

# A Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Explanation

1 Jan 2005 8:00 UTC
45 points

# Bayes’ The­o­rem Illus­trated (My Way)

3 Jun 2010 4:40 UTC
137 points

# It’s Bayes All The Way Up

12 Sep 2016 13:35 UTC
20 points

# The Equa­tion of Knowledge

7 Jul 2020 16:09 UTC
59 points

# Bayes for Schizophren­ics: Rea­son­ing in Delu­sional Disorders

13 Aug 2012 19:22 UTC
96 points

# What is Bayesi­anism?

26 Feb 2010 7:43 UTC
87 points

# Learn Bayes Nets!

27 Mar 2018 22:00 UTC
85 points

# Bayes’ Law is About Mul­ti­ple Hy­poth­e­sis Testing

4 May 2018 5:31 UTC
83 points

# Bayesian examination

9 Dec 2019 19:50 UTC
86 points

# Bayesian Prob­a­bil­ity is for things that are Space-like Separated from You

10 Jul 2018 23:47 UTC
79 points

# What Bayesi­anism taught me

12 Aug 2013 6:59 UTC
70 points

# Kal­man Filter for Bayesians

22 Oct 2018 17:06 UTC
59 points

# Bayesi­anism for Humans

29 Oct 2013 23:54 UTC
57 points

# Against strong bayesianism

30 Apr 2020 10:48 UTC
49 points

# Bayes’ rule =/​= Bayesian inference

16 Sep 2010 6:34 UTC
39 points

# Bayes Rule Applied

16 Feb 2018 18:30 UTC
12 points
(towardsdatascience.com)

# [Question] al­ter­na­tive his­tory: what if Bayes rule had never been dis­cov­ered?

11 May 2019 7:29 UTC
7 points

# nos­talge­braist—bayes: a kinda-sorta masterpost

4 Sep 2018 11:08 UTC
24 points
(nostalgebraist.tumblr.com)

# Fal­la­cies as weak Bayesian evidence

18 Mar 2012 3:53 UTC
58 points

# An­thropic rea­son­ing isn’t magic

1 Nov 2017 8:57 UTC
36 points

# Kelly bettors

13 Nov 2018 0:40 UTC
23 points
(danielfilan.com)

# Yes, Virginia, You Can Be 99.99% (Or More!) Cer­tain That 53 Is Prime

7 Nov 2013 7:45 UTC
52 points

# A His­tory of Bayes’ Theorem

29 Aug 2011 7:04 UTC
57 points

26 Jun 2020 22:14 UTC
45 points

# Mini-re­view: ‘Prov­ing His­tory: Bayes’ The­o­rem and the Quest for the His­tor­i­cal Je­sus’

1 Feb 2012 19:20 UTC
22 points

# Search­ing for Bayes-Structure

28 Feb 2008 22:01 UTC
37 points

# How to Mea­sure Anything

7 Aug 2013 4:05 UTC
71 points

# Refer­ences & Re­sources for LessWrong

10 Oct 2010 14:54 UTC
123 points

# Bayes Questions

7 Nov 2018 16:54 UTC
22 points

# [Question] What are prin­ci­pled ways for pe­nal­is­ing com­plex­ity in prac­tice?

27 Jun 2019 7:28 UTC
42 points

# Prob­a­bil­ity is in the Mind

12 Mar 2008 4:08 UTC
87 points

# Bayesian Evolv­ing-to-Extinction

14 Feb 2020 23:55 UTC
39 points

# Non-com­mu­ni­ca­ble Evidence

17 Nov 2015 3:46 UTC
10 points

# Qual­i­ta­tively Confused

14 Mar 2008 17:01 UTC
38 points

# My Bayesian Enlightenment

5 Oct 2008 16:45 UTC
34 points

# Beau­tiful Probability

14 Jan 2008 7:19 UTC
59 points

# De­co­her­ence is Falsifi­able and Testable

7 May 2008 7:54 UTC
33 points

# Prob­a­bil­ity Space & Au­mann Agreement

10 Dec 2009 21:57 UTC
44 points

# Au­mann’s Agree­ment Revisited

27 Aug 2018 6:21 UTC
4 points

# The Me­chan­ics of Disagreement

10 Dec 2008 14:01 UTC
8 points

# The Joys of Con­ju­gate Priors

21 May 2011 2:41 UTC
43 points

# Bayesi­anism for hu­mans: “prob­a­ble enough”

2 Sep 2014 21:44 UTC
38 points

# Nav­i­gat­ing dis­agree­ment: How to keep your eye on the ev­i­dence

24 Apr 2010 22:47 UTC
37 points

# If It’s Worth Do­ing, It’s Worth Do­ing With Made-Up Statistics

3 Sep 2017 20:56 UTC
36 points

# Beyond Bayesi­ans and Frequentists

31 Oct 2012 7:03 UTC
36 points

# Born as the sev­enth month dies …

10 Jul 2020 15:07 UTC
6 points

# [Question] Can Bayes the­o­rem rep­re­sent in­finite con­fu­sion?

22 Mar 2019 18:02 UTC
4 points

# Why We Can’t Take Ex­pected Value Es­ti­mates Liter­ally (Even When They’re Un­bi­ased)

18 Aug 2011 23:34 UTC
95 points

# Fre­quen­tist Statis­tics are Fre­quently Subjective

4 Dec 2009 20:22 UTC
71 points

# Dou­ble Illu­sion of Transparency

24 Oct 2007 23:06 UTC
66 points

# Against NHST

21 Dec 2012 4:45 UTC
63 points

# In­ter­pre­ta­tions of “prob­a­bil­ity”

9 May 2019 19:16 UTC
69 points

# Ein­stein’s Arrogance

25 Sep 2007 1:29 UTC
67 points

# A Fer­vent Defense of Fre­quen­tist Statistics

18 Feb 2014 20:08 UTC
47 points

# In­finite Certainty

9 Jan 2008 6:49 UTC
50 points

# Fre­quen­tist Magic vs. Bayesian Magic

8 Apr 2010 20:34 UTC
43 points

# How Much Ev­i­dence Does It Take?

24 Sep 2007 4:06 UTC
61 points

# Bayes Academy: Devel­op­ment re­port 1

19 Nov 2014 22:35 UTC
47 points

# Multiplicitous

18 Dec 2016 16:39 UTC
9 points
(putanumonit.com)

# Dreams with Da­m­aged Priors

8 Aug 2009 22:31 UTC
39 points

# Bayes’ The­o­rem in three pictures

21 Jul 2019 7:01 UTC
32 points