A: Mathematicians broken down by age and sex.
Q: What are Bayesian statisticians?
A: Mathematicians broken down by age, sex, and level of alcohol consumption.
|(Above jokes and picture from the website "Bayesian Fun", click here)|
What is the ontological status of probability? That is, to what extent does it really exist in the world? There are two broad schools of thought: Frequentism and Bayesianism.
For a frequentist, randomness is taken as an intrinsic part of reality, which probability quantifies. To say that event A has a probability of one half means that if the experiment was repeated many times then A would occur exactly one half of the time. In other words the probability of A is a measure of the frequency with which A happens, given the initial conditions. (This would only be an approximate after finitely many repetitions, but would be exact in the limit.)
As this shows, the principle does not apply very easily to one-off events, but is best suited to repetitive occurrences.
In contrast to frequentists, for a Bayesian, probability does not exist in the external world. It is purely a way for humans to quantify our degree of certainty on the basis of incomplete information. In other words, probability is a subjective concept. People will make different assessments of probability, based on the different data they have available.
So if a coin flip is initially judged to have a probability of one-half of resulting in a head, this is because we know little about it. More data about the weighing of the coin, its initial position, and the technique of the flipper would allow us to modify our probability. If we knew these things in great detail, we would be able to predict the outcome with some certainty. (The mathematician John Conway is reputed to have mastered the art of flipping coins to order.)
There is a consequence to the Bayesian view. To a Bayesian, all probability is conditional. Suppose you estimate the probability of A happening as P(A). (This is really P(A/C) where C represents your current knowledge, but we suppress this). This is your prior probability. When some new data (B) comes to light, you need to update this assessment. This means using conditional probability to calculate P(A/B), called your posterior probability.
As the fallacy of probability inversion shows, Bayesian interference can throw up counter-intuitive results. Bayesian thinkers deploy this technique to improve probability assessment in a broad range of subjects, from economics to artificial intelligence.
From: Mathematics 1001 by Dr. Richard Elwes