Bayesian inference

Template:Bayesian statistics Bayesian inference (Template:IPAc-en Template:Respell or Template:IPAc-en Template:Respell)Template:Refn is a type of statistical inference. In Bayesian inference, evidence or information is available, Bayes' theorem is used to change (or update) the probability of a hypothesis. Bayesian inference uses a prior distribution to estimate posterior probabilities. Bayesian inference is an important to statistics, mathematical statistics, decision theory, and sequential analysis. Bayesian inference is used in science, engineering, philosophy, medicine, sport, and law.

Bayes' rule

A geometric visualisation of Bayes' theorem. In the table, the values 2, 3, 6 and 9 give the relative weights of each corresponding condition and case. The figures denote the cells of the table involved in each metric, the probability being the fraction of each figure that is shaded. This shows that P(A|B) P(B) = P(B|A) P(A) i.e. P(A|B) = Template:Sfrac . Similar reasoning can be used to show that P(¬A|B) = Template:Sfrac etc.

Contingency table

Template:Diagonal split header	Satisfies hypothesis H	Violates hypothesis ¬H		Total
Has evidence E	P(H|E)·P(E) = P(E|H)·P(H)	P(¬H|E)·P(E) = P(E|¬H)·P(¬H)		P(E)
No evidence ¬E	P(H|¬E)·P(¬E) = P(¬E|H)·P(H)	P(¬H|¬E)·P(¬E) = P(¬E|¬H)·P(¬H)		P(¬E) = 1−P(E)
Total	P(H)	P(¬H) = 1−P(H)	1

Bayesian inference figures out the posterior probability from prior probability and the "likelihood function". The likelihood function comes from a statistical model of the data. $$P(H\mid E)=\frac{P(E\mid H)\cdot P(H)}{P(E)},$$ where

• $H$ is a hypothesis that is changed by data (or evidence). There are usually many hypotheses. The point of the test is to see which hypothesis is more likely.
• $P(H)$ is the prior probability. It estimates the probability of a hypothesis before there is any evidence.
• $E$ is the evidence, or data. It is any new data that is found.
• $P(H\mid E)$ is the posterior probability. This is what we want to know.
• $P(E\mid H)$ is the likelihood function.
• $P(E)$ is the marginal likelihood. It is the same for all possible hypotheses that are being tested. $P(E)$ has to be greater than 0. If $P(E)$ is 0, then you divide by zero.

Related pages

• Epistemology
• Free energy principle

Further reading

• Template:Cite book
• Template:Cite book

• Stone, JV (2013), "Bayes' Rule: A Tutorial Introduction to Bayesian Analysis", Download first chapter here, Sebtel Press, England.
• Template:Cite book
• Template:Cite book
• Template:Cite book
• Template:Cite book
• Bolstad, William M. (2007) Introduction to Bayesian Statistics: Second Edition, John Wiley Template:ISBN
• Template:Cite book Updated classic textbook. Bayesian theory clearly presented.
• Lee, Peter M. Bayesian Statistics: An Introduction. Fourth Edition (2012), John Wiley Template:ISBN
• Template:Cite book
• Template:Cite book

• Template:Cite book
• Template:Cite book
• DeGroot, Morris H., Optimal Statistical Decisions. Wiley Classics Library. 2004. (Originally published (1970) by McGraw-Hill.) Template:ISBN.
• Template:Cite book
• Jaynes, E. T. (1998). Probability Theory: The Logic of Science.
• O'Hagan, A. and Forster, J. (2003). Kendall's Advanced Theory of Statistics, Volume 2B: Bayesian Inference. Arnold, New York. Template:ISBN.
• Template:Cite book
• Pearl, Judea. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, San Mateo, CA: Morgan Kaufmann.
• Pierre Bessière et al. (2013). "Bayesian Programming". CRC Press. Template:ISBN
• Francisco J. Samaniego (2010). "A Comparison of the Bayesian and Frequentist Approaches to Estimation". Springer. New York, Template:ISBN

Other websites

• Bayesian Statistics from Scholarpedia.
• Introduction to Bayesian probability from Queen Mary University of London
• Mathematical Notes on Bayesian Statistics and Markov Chain Monte Carlo
• Bayesian reading list Template:Webarchive, categorized and annotated by Tom Griffiths
• A. Hajek and S. Hartmann: Bayesian Epistemology, in: J. Dancy et al. (eds.), A Companion to Epistemology. Oxford: Blackwell 2010, 93–106.
• S. Hartmann and J. Sprenger: Bayesian Epistemology, in: S. Bernecker and D. Pritchard (eds.), Routledge Companion to Epistemology. London: Routledge 2010, 609–620.
• Stanford Encyclopedia of Philosophy: "Inductive Logic"
• Bayesian Confirmation Theory (PDF)
• Data, Uncertainty and Inference — Informal introduction with many examples, ebook (PDF) freely available at causaScientia
• Broad Introduction to Bayesian Statistics for Machine Learning