Bayesian inference

key terms

To understand Bayesian inference, it's important to understand the following key terms:

Prior probability: This is the probability of an event or parameter before we have observed any data or evidence. It reflects our initial beliefs, knowledge, or assumptions about the event or parameter.

Likelihood: This is the probability of the observed data or evidence given the event or parameter. It reflects the degree of compatibility between the data and the event or parameter.

Posterior probability: This is the probability of the event or parameter given the observed data or evidence. It reflects our updated beliefs or knowledge about the event or parameter after taking the data into account. In other words, the posterior probability is what we are interested in calculating in Bayesian inference.

Marginal probability: This is the probability of the observed data or evidence, regardless of the value of the event or parameter. It is obtained by integrating the joint probability of the data and the event or parameter over all possible values of the event or parameter. The marginal probability is a normalization factor in Bayes' theorem and ensures that the posterior probability is a valid probability distribution.

To illustrate these concepts, let's consider a simple example of flipping a coin. Suppose we have a coin that we suspect might be biased towards heads. We assign a prior probability of 0.6 to the coin landing on heads, which reflects our initial belief. We then flip the coin 10 times and observe that it lands on heads 7 times and tails 3 times. We can calculate the likelihood of the data given the event that the coin lands on heads with probability 0.6 as follows:

Likelihood = P(data|event) = (0.6)^7 * (0.4)^3 = 0.042467

Next, we use Bayes' theorem to update our prior probability with the likelihood to obtain the posterior probability:

Posterior probability = P(event|data) = P(data|event) * P(event) / P(data)

where P(data) is the marginal probability, which we can calculate by summing over all possible values of the event:

P(data) = P(data|event=H) * P(event=H) + P(data|event=T) * P(event=T)
= (0.6)^7 * (0.4)^3 * 0.6 + (0.6)^3 * (0.4)^7 * 0.4
= 0.03535 + 0.00066
= 0.03601

Plugging in the values, we get:

Posterior probability = P(event|data) = P(data|event) * P(event) / P(data)
= 0.042467 * 0.6 / 0.03601
= 0.708

Therefore, the posterior probability that the coin lands on heads with probability 0.6 is 0.708 after observing the data. This reflects our updated belief or knowledge about the coin after taking the data into account.

 

Bayesian inference

Bayesian inference is a statistical method that allows us to make predictions or inferences about uncertain events or parameters based on prior knowledge or data. It is named after the statistician Thomas Bayes, who first formulated the principles of Bayesian inference in the 18th century.

 

The basic idea behind Bayesian inference is to use Bayes' theorem to update our prior beliefs or knowledge about a parameter or event based on new evidence or data. Bayes' theorem states that the probability of an event or parameter given the data is proportional to the probability of the data given the event or parameter, multiplied by the prior probability of the event or parameter.

 

Mathematically, Bayes' theorem can be written as:

P(A|B) = P(B|A) * P(A) / P(B)

Where:

• P(A|B) is the probability of A given B, which is the posterior probability
• P(B|A) is the probability of B given A, which is the likelihood
• P(A) is the prior probability of A
• P(B) is the marginal probability of B

To apply Bayesian inference, we start with a prior probability distribution that represents our initial beliefs or knowledge about the parameter or event. Then, we update this prior distribution using the data or evidence to obtain the posterior distribution, which represents our updated beliefs or knowledge.

 

The process of Bayesian inference involves the following steps:

1. Define the prior distribution: This involves choosing a probability distribution that represents our prior beliefs or knowledge about the parameter or event.
2. Collect data: We collect data or evidence related to the parameter or event.
3. Calculate the likelihood: We calculate the likelihood of the data given the parameter or event, which represents the probability of obtaining the observed data given the parameter or event.
4. Calculate the posterior distribution: We use Bayes' theorem to update our prior distribution with the likelihood and obtain the posterior distribution, which represents our updated beliefs or knowledge about the parameter or event.
5. Evaluate the results: We evaluate the posterior distribution to make predictions or draw conclusions about the parameter or event.

Bayesian inference has several advantages over other statistical methods, including the ability to incorporate prior knowledge and the ability to update our beliefs as new data becomes available. However, it can be computationally intensive and requires careful consideration of the choice of prior distribution.