Bayes’ Theorem

TagLast edit: 19 Feb 2024 5:40 UTC by

Bayes’ Theorem (also known as Bayes’ Law) is a law of probability that describes the proper way to incorporate new evidence into prior probabilities to form an updated probability estimate. It is commonly regarded as the foundation of consistent rational reasoning under uncertainty. Bayes Theorem is named after Reverend Thomas Bayes who proved the theorem in 1763.

Bayes’ theorem commonly takes the form:

where A is the proposition of interest, B is the observed evidence, P(A) and P(B) are prior probabilities, and P(A|B) is the posterior probability of A.

With the posterior odds, the prior odds and the likelihood ratio written explicitly, the theorem reads:

An In­tu­itive Ex­pla­na­tion of Bayes’s Theorem

1 Jan 2003 20:00 UTC
63 points

18 Aug 2020 21:14 UTC
176 points

A Tech­ni­cal Ex­pla­na­tion of Tech­ni­cal Explanation

1 Jan 2005 8:00 UTC
68 points

a vi­sual ex­pla­na­tion of Bayesian updating

8 May 2021 19:45 UTC
20 points

It’s Bayes All The Way Up

12 Sep 2016 13:35 UTC
31 points

The Moun­tain Troll

11 Jun 2022 9:14 UTC
96 points

Anec­dotes Can Be Strong Ev­i­dence and Bayes The­o­rem Proves It

13 Mar 2022 4:06 UTC
16 points

Bayes’ The­o­rem Illus­trated (My Way)

3 Jun 2010 4:40 UTC
169 points

The Equa­tion of Knowledge

7 Jul 2020 16:09 UTC
58 points

22 Nov 2021 10:39 UTC
79 points

Towards a Bayesian model for Em­piri­cal Science

7 Oct 2021 5:38 UTC
11 points

Bayesian Utility: Rep­re­sent­ing Prefer­ence by Prob­a­bil­ity Measures

27 Jul 2009 14:28 UTC
48 points

E.T. Jaynes Prob­a­bil­ity The­ory: The logic of Science I

27 Dec 2023 23:47 UTC
49 points

Strong Ev­i­dence is Common

13 Mar 2021 22:04 UTC
242 points
(markxu.com)

Bayesian Evolv­ing-to-Extinction

14 Feb 2020 23:55 UTC
38 points

Bayes for Schizophren­ics: Rea­son­ing in Delu­sional Disorders

13 Aug 2012 19:22 UTC
151 points

What is Bayesi­anism?

26 Feb 2010 7:43 UTC
115 points

Learn Bayes Nets!

27 Mar 2018 22:00 UTC
50 points

Bayes’ Law is About Mul­ti­ple Hy­poth­e­sis Testing

4 May 2018 5:31 UTC
36 points

Bayesian examination

9 Dec 2019 19:50 UTC
86 points

Non-com­mu­ni­ca­ble Evidence

17 Nov 2015 3:46 UTC
17 points

Bayesian Prob­a­bil­ity is for things that are Space-like Separated from You

10 Jul 2018 23:47 UTC
80 points

What can we learn from Bayes about rea­son­ing?

5 May 2023 15:52 UTC
21 points

What Bayesi­anism taught me

12 Aug 2013 6:59 UTC
115 points

Kal­man Filter for Bayesians

22 Oct 2018 17:06 UTC
51 points

Bayesi­anism for Humans

29 Oct 2013 23:54 UTC
91 points

Bayes’ the­o­rem, plau­si­ble de­ni­a­bil­ity, and smiley faces

11 Apr 2021 20:41 UTC
24 points

Against strong bayesianism

30 Apr 2020 10:48 UTC
58 points

Qual­i­ta­tively Confused

14 Mar 2008 17:01 UTC
59 points

Black ravens and red herrings

27 Jul 2021 17:46 UTC
50 points

Bayes’ rule =/​= Bayesian inference

16 Sep 2010 6:34 UTC
49 points

Bayes’ The­o­rem in three pictures

21 Jul 2019 7:01 UTC
30 points

Base Rates and Refer­ence Classes

24 Nov 2021 22:30 UTC
20 points
(bounded-regret.ghost.io)

[Question] How do Bayesi­ans tell what does and doesn’t count as ev­i­dence (which, e.g., hy­pothe­ses may ren­der more or less prob­a­ble if true)? Is it pos­si­ble for some­thing to fuzzily-be ev­i­dence?

30 Nov 2021 12:52 UTC
2 points

My Bayesian Enlightenment

5 Oct 2008 16:45 UTC
67 points

Beau­tiful Probability

14 Jan 2008 7:19 UTC
101 points

De­co­her­ence is Falsifi­able and Testable

7 May 2008 7:54 UTC
44 points

Ep­silon is not a prob­a­bil­ity, it’s a cop-out

15 Feb 2022 2:48 UTC
23 points

The Case for Fre­quen­tism: Why Bayesian Prob­a­bil­ity is Fun­da­men­tally Un­sound and What Science Does Instead

3 Apr 2022 20:52 UTC
22 points

Bayes Rule Applied

16 Feb 2018 18:30 UTC
4 points
(towardsdatascience.com)

Re­port like­li­hood ratios

23 Apr 2022 17:10 UTC
80 points

Un­der­stand­ing Sub­jec­tive Probabilities

10 Dec 2023 6:03 UTC
30 points

Prob­a­bil­ity Space & Au­mann Agreement

10 Dec 2009 21:57 UTC
52 points

[Link] New SEP ar­ti­cle on Bayesian Epistemology

13 Jun 2022 15:03 UTC
6 points

Au­mann’s Agree­ment Revisited

27 Aug 2018 6:21 UTC
4 points

The Me­chan­ics of Disagreement

10 Dec 2008 14:01 UTC
14 points

The Joys of Con­ju­gate Priors

21 May 2011 2:41 UTC
63 points

“Friends do not let friends com­pute p val­ues.”

9 Sep 2011 7:10 UTC
15 points

Bayesi­anism for hu­mans: “prob­a­ble enough”

2 Sep 2014 21:44 UTC
52 points

Nav­i­gat­ing dis­agree­ment: How to keep your eye on the ev­i­dence

24 Apr 2010 22:47 UTC
47 points

If It’s Worth Do­ing, It’s Worth Do­ing With Made-Up Statistics

3 Sep 2017 20:56 UTC
68 points

When ap­par­ently pos­i­tive ev­i­dence can be nega­tive evidence

20 Oct 2022 21:47 UTC
19 points
(www.ncbi.nlm.nih.gov)

Beyond Bayesi­ans and Frequentists

31 Oct 2012 7:03 UTC
55 points

Creat­ing a database for base rates

12 Dec 2022 10:09 UTC
2 points
(forum.effectivealtruism.org)

[Question] al­ter­na­tive his­tory: what if Bayes rule had never been dis­cov­ered?

11 May 2019 7:29 UTC
7 points

The role of Bayesian ML in AI safety—an overview

27 Jan 2023 19:40 UTC
30 points

Prob­a­bil­ity The­ory: The Logic of Science, Jaynes

16 Feb 2023 21:57 UTC
29 points

Bayesian Sce­nario: Snipers & Soldiers

26 Feb 2023 21:48 UTC
23 points
(h-b-p.github.io)

Fun­da­men­tal Uncer­tainty: Chap­ter 3 - Why don’t we agree on what’s right?

25 Jun 2022 17:50 UTC
27 points

nos­talge­braist—bayes: a kinda-sorta masterpost

4 Sep 2018 11:08 UTC
25 points
(nostalgebraist.tumblr.com)

Fal­la­cies as weak Bayesian evidence

18 Mar 2012 3:53 UTC
87 points

An­thropic rea­son­ing isn’t magic

1 Nov 2017 8:57 UTC
22 points

Kelly bettors

13 Nov 2018 0:40 UTC
24 points
(danielfilan.com)

Yes, Virginia, You Can Be 99.99% (Or More!) Cer­tain That 53 Is Prime

7 Nov 2013 7:45 UTC
95 points

Ab­solute Authority

8 Jan 2008 3:33 UTC
94 points

In­ter­ac­tive Bayes The­o­rem Visualization

3 Mar 2018 4:00 UTC
14 points
(allenkim67.github.io)

Very Short In­tro­duc­tion to Bayesian Model Com­par­i­son

16 Jul 2019 19:48 UTC
30 points

A His­tory of Bayes’ Theorem

29 Aug 2011 7:04 UTC
75 points

Are Bayesian meth­ods guaran­teed to overfit?

17 Jun 2023 12:52 UTC
52 points
(www.yulingyao.com)

26 Jun 2020 22:14 UTC
48 points

Mini-re­view: ‘Prov­ing His­tory: Bayes’ The­o­rem and the Quest for the His­tor­i­cal Je­sus’

1 Feb 2012 19:20 UTC
26 points

In­tro­duc­ing bayescalc.io

7 Jul 2023 16:11 UTC
114 points
(bayescalc.io)

Search­ing for Bayes-Structure

28 Feb 2008 22:01 UTC
60 points

How to Mea­sure Anything

7 Aug 2013 4:05 UTC
117 points

Refer­ences & Re­sources for LessWrong

10 Oct 2010 14:54 UTC
162 points

Bayes Questions

7 Nov 2018 16:54 UTC
21 points

[Question] What are prin­ci­pled ways for pe­nal­is­ing com­plex­ity in prac­tice?

27 Jun 2019 7:28 UTC
36 points

Prob­a­bil­ity is in the Mind

12 Mar 2008 4:08 UTC
129 points

The Bayesian Tyrant

20 Aug 2020 0:08 UTC
140 points

Why Bayes? A Wise Ruling

25 Feb 2013 15:52 UTC
18 points

Boltz­mann brain’s con­di­tional probability

29 Dec 2023 14:44 UTC
5 points

Bayesi­ans Com­mit the Gam­bler’s Fallacy

7 Jan 2024 12:54 UTC
46 points
(kevindorst.substack.com)

How do you ac­tu­ally ob­tain and re­port a like­li­hood func­tion for sci­en­tific re­search?

11 Feb 2024 17:42 UTC
54 points

Not us­ing a pri­ori in­for­ma­tion for Rus­sian propaganda

24 Apr 2023 1:14 UTC
0 points

Born as the sev­enth month dies …

10 Jul 2020 15:07 UTC
6 points

[Question] Can Bayes the­o­rem rep­re­sent in­finite con­fu­sion?

22 Mar 2019 18:02 UTC
4 points

Why We Can’t Take Ex­pected Value Es­ti­mates Liter­ally (Even When They’re Un­bi­ased)

18 Aug 2011 23:34 UTC
123 points

Fre­quen­tist Statis­tics are Fre­quently Subjective

4 Dec 2009 20:22 UTC
86 points

Dou­ble Illu­sion of Transparency

24 Oct 2007 23:06 UTC
113 points

Against NHST

21 Dec 2012 4:45 UTC
93 points

In­ter­pre­ta­tions of “prob­a­bil­ity”

9 May 2019 19:16 UTC
68 points

Ein­stein’s Arrogance

25 Sep 2007 1:29 UTC
150 points

A Fer­vent Defense of Fre­quen­tist Statistics

18 Feb 2014 20:08 UTC
74 points

In­finite Certainty

9 Jan 2008 6:49 UTC
75 points

Fre­quen­tist Magic vs. Bayesian Magic

8 Apr 2010 20:34 UTC
58 points

How Much Ev­i­dence Does It Take?

24 Sep 2007 4:06 UTC
126 points

Multiplicitous

18 Dec 2016 16:39 UTC
9 points
(putanumonit.com)

19 Nov 2014 22:35 UTC
68 points

Dreams with Da­m­aged Priors

8 Aug 2009 22:31 UTC
59 points

[Question] Bayesian up­date from sen­sa­tion­al­is­tic sources

11 May 2023 15:26 UTC
1 point

Hearsay, Dou­ble Hearsay, and Bayesian Updates

16 Feb 2012 22:31 UTC
68 points

Prob­a­bil­ity is Sub­jec­tively Objective

14 Jul 2008 9:16 UTC
40 points

Pri­ors as Math­e­mat­i­cal Objects

12 Apr 2007 3:24 UTC
51 points

Bayesian Flame

26 Jul 2009 16:49 UTC
41 points

[Funny] Even Clippy can be blamed on the use of non-Bayesian methods

2 Oct 2011 7:40 UTC
41 points

When (Not) To Use Probabilities

23 Jul 2008 10:58 UTC
63 points

The Quick Bayes Table

18 Apr 2012 18:00 UTC
60 points

In­tro­spec­tive Bayes

27 May 2023 19:35 UTC
−5 points

De­mands for Par­tic­u­lar Proof: Appendices

15 Feb 2010 7:58 UTC
39 points

(Sub­jec­tive Bayesi­anism vs. Fre­quen­tism) VS. For­mal­ism

26 Nov 2011 5:05 UTC
32 points

Laplace Approximation

18 Jul 2019 15:23 UTC
28 points

Knigh­tian un­cer­tainty in a Bayesian framework

24 Jul 2014 14:31 UTC
54 points

In­her­ited Im­prob­a­bil­ities: Trans­fer­ring the Bur­den of Proof

24 Nov 2010 3:40 UTC
46 points

Knigh­tian un­cer­tainty: a re­jec­tion of the MMEU rule

26 Aug 2014 3:03 UTC
41 points

The prior of a hy­poth­e­sis does not de­pend on its complexity

26 Aug 2010 13:20 UTC
34 points

RFC on an open prob­lem: how to de­ter­mine prob­a­bil­ities in the face of so­cial distortion

7 Oct 2017 22:04 UTC
6 points

In­tro­duc­tion To The In­fra-Bayesi­anism Sequence

26 Aug 2020 20:31 UTC
108 points

Fre­quen­tist vs Bayesian break­down: in­ter­pre­ta­tion vs inference

30 Aug 2011 15:58 UTC
36 points

[Question] Why is Bayesi­anism im­por­tant for ra­tio­nal­ity?

1 Sep 2020 4:24 UTC
37 points

Weak Ev­i­dence is Common

16 Jul 2023 23:37 UTC
7 points
(dkl9.net)

Embed­ding Eth­i­cal Pri­ors into AI Sys­tems: A Bayesian Approach

3 Aug 2023 15:31 UTC
−5 points

Evolved Bayesi­ans will be biased

20 Aug 2009 14:54 UTC
28 points

30 Jun 2010 17:21 UTC
37 points

Lev­er­ag­ing Bayes’ The­o­rem to Su­per­charge Me­mory Techniques

9 Oct 2023 3:34 UTC
−15 points

Dutch Books and De­ci­sion The­ory: An In­tro­duc­tion to a Long Conversation

21 Dec 2010 4:55 UTC
30 points

LINK: Bayesian statis­tics is so sub­ver­sive, it’s banned in China!

11 Jan 2011 0:50 UTC
4 points

The med­i­cal test para­dox: Can re­design­ing Bayes rule help?

24 Dec 2020 22:08 UTC
34 points

Con­ser­va­tion of Ex­pected Evidence

13 Aug 2007 15:55 UTC
238 points

[Question] What’s the big deal about Bayes’ The­o­rem?

26 Jan 2021 6:08 UTC
21 points

Bayesian in­fer­ence on 1st or­der logic

3 Feb 2021 1:03 UTC
11 points

An In­tu­itive Ex­pla­na­tion of Eliezer Yud­kowsky’s In­tu­itive Ex­pla­na­tion of Bayes’ Theorem

18 Dec 2010 13:26 UTC
24 points

Par­ti­ci­pat­ing in a Covid-19 Vac­cine Trial #3: I Hope I feel Worse Tomorrow

5 Mar 2021 0:02 UTC
8 points

11 Mar 2021 8:11 UTC
18 points

The Ja­panese Quiz: a Thought Ex­per­i­ment of Statis­ti­cal Epistemology

8 Apr 2021 17:37 UTC
11 points

Book Re­view of 5 Ap­plied Bayesian Statis­tics Books

21 May 2021 10:23 UTC
66 points

Dooms­day, Sam­pling As­sump­tions, and Bayes

14 Jun 2021 12:41 UTC
6 points

In­ter­pret­ing Di­ag­nos­tic Tests

15 Oct 2021 2:58 UTC
6 points

XKCD—Fre­quen­tist vs. Bayesians

9 Nov 2012 5:25 UTC
24 points

Find­ing the Cen­tral Limit The­o­rem in Bayes’ rule

27 Nov 2021 5:48 UTC
22 points

Buck­ets & Bayes

20 Dec 2021 5:54 UTC
34 points

[Question] How would you ex­plain Bayesian think­ing to a ten year old?

5 Jan 2022 17:25 UTC
7 points

Ret­ro­spec­tive forecasting

30 Jan 2022 16:38 UTC
21 points

Com­men­su­rable Scien­tific Paradigms; or, com­putable induction

13 Apr 2022 0:01 UTC
14 points

RL with KL penalties is bet­ter seen as Bayesian inference

25 May 2022 9:23 UTC
114 points

How to Vi­su­al­ize Bayesianism

22 Jun 2022 13:57 UTC
9 points

King David’s %: Estab­lish­ing a new sym­bol for Bayesian prob­a­bil­ity.

26 Jun 2022 19:47 UTC
−11 points
(laulpogan.substack.com)

Five views of Bayes’ Theorem

2 Jul 2022 2:25 UTC
38 points

An at­tempt to break cir­cu­lar­ity in science

15 Jul 2022 18:32 UTC
3 points

Bayesian prob­a­bil­ity the­ory as ex­tended logic—a new result

6 Jul 2017 19:14 UTC
37 points

Think­ing with­out pri­ors?

2 Aug 2022 9:17 UTC
7 points

Mak­ing Vaccine

3 Feb 2021 20:24 UTC
569 points

Log-odds are bet­ter than Probabilities

12 Dec 2022 20:10 UTC
22 points