Bayesian Theorem

What Is Bayes’ Theorem?

Tests are not the event. We have a cancer test, separate from the event of actually having cancer. We have a test for spam, separate from the event of actually having a spam message.

Tests are flawed. Tests detect things that don’t exist (false positive), and miss things that do exist (false negative). People often use test results without adjusting for test errors.

False positives skew results. Suppose you are searching for something really rare (1 in a million). Even with a good test, it’s likely that a positive result is really a false positive on somebody in the 999,999.

People prefer natural numbers. Saying “100 in 10,000″ rather than “1%” helps people work through the numbers with fewer errors, especially with multiple percentages (“Of those 100, 80 will test positive” rather than “80% of the 1% will test positive”).

Even science is a test. At a philosophical level, scientific experiments are “potentially flawed tests” and need to be treated accordingly. There is a test for a chemical, or a phenomenon, and there is the event of the phenomenon itself. Our tests and measuring equipment have a rate of error to be accounted for.

Bayes’ theorem converts the results from your test into the real probability of the event. For example, you can:

  • Correct for measurement errors. If you know the real probabilities and the chance of a false positive and false negative, you can correct for measurement errors.
  • Relate the actual probability to the measured test probability. Given mammogram test results and known error rates, you can predict the actual chance of having cancer given a positive test. In technical terms, you can find Pr(H|E), the chance that a hypothesis H is true given evidence E, starting from Pr(E|H), the chance that evidence appears when the hypothesis is true.

Bayes’ theorem, named after 18th-century British mathematician Thomas Bayes, is a mathematical formula for determining conditional probability. Conditional probability is the likelihood of an outcome occurring, based on a previous outcome occurring. Bayes’ theorem provides a way to revise existing predictions or theories (update probabilities) given new or additional evidence. In finance, Bayes’ theorem can be used to rate the risk of lending money to potential borrowers.

Bayes’ theorem is also called Bayes’ Rule or Bayes’ Law and is the foundation of the field of Bayesian statistics.

Many modern machine learning techniques rely on Bayes’ theorem. For instance, spam filters use Bayesian updating to determine whether an email is real or spam, given the words in the email. Additionally, many specific techniques in statistics, such as calculating ppp-values or interpreting medical results, are best described in terms of how they contribute to updating hypotheses using Bayes’ theorem.

Key Takeaways

  • Bayes’ theorem allows you to update predicted probabilities of an event by incorporating new information.
  • Bayes’ theorem was named after 18th-century mathematician Thomas Bayes.
  • It is often employed in finance in updating risk evaluation.

Understanding Bayes’ Theorem

Applications of the theorem are widespread and not limited to the financial realm. As an example, Bayes’ theorem can be used to determine the accuracy of medical test results by taking into consideration how likely any given person is to have a disease and the general accuracy of the test. Bayes’ theorem relies on incorporating prior probability distributions in order to generate posterior probabilities. Prior probability, in Bayesian statistical inference, is the probability of an event before new data is collected. This is the best rational assessment of the probability of an outcome based on the current knowledge before an experiment is performed. Posterior probability is the revised probability of an event occurring after taking into consideration new information. Posterior probability is calculated by updating the prior probability by using Bayes’ theorem. In statistical terms, the posterior probability is the probability of event A occurring given that event B has occurred.

Bayes’ theorem thus gives the probability of an event based on new information that is, or may be related, to that event. The formula can also be used to see how the probability of an event occurring is affected by hypothetical new information, supposing the new information will turn out to be true. For instance, say a single card is drawn from a complete deck of 52 cards. The probability that the card is a king is four divided by 52, which equals 1/13 or approximately 7.69%. Remember that there are four kings in the deck. Now, suppose it is revealed that the selected card is a face card. The probability the selected card is a king, given it is a face card, is four divided by 12, or approximately 33.3%, as there are 12 face cards in a deck.

Examples of Bayes’ Theorem

Below are two examples of Bayes’ theorem in which the first example shows how the formula can be derived in a stock investing example using Inc. (AMZN). The second example applies Bayes’ theorem to pharmaceutical drug testing.

Deriving the Bayes’ Theorem Formula

Bayes’ theorem follows simply from the axioms of conditional probability. Conditional probability is the probability of an event given that another event occurred. For example, a simple probability question may ask: “What is the probability of’s stock price falling?” Conditional probability takes this question a step further by asking: “What is the probability of AMZN stock price falling given that the Dow Jones Industrial Average (DJIA) index fell earlier?”

The conditional probability of A given that B has happened can be expressed as:

If A is: “AMZN price falls” then P(AMZN) is the probability that AMZN falls; and B is: “DJIA is already down,” and P(DJIA) is the probability that the DJIA fell; then the conditional probability expression reads as “the probability that AMZN drops given a DJIA decline is equal to the probability that AMZN price declines and DJIA declines over the probability of a decrease in the DJIA index.


What’s the Bayesian Trap?