Introduction

So far in this course, nearly every statistical model we have studied has been based on the frequentist framework.

Examples:

• Linear Regression
• Logistic Regression
• Poisson Regression
• Negative Binomial Regression
• Mixed Models
• Survival Analysis

In frequentist statistics:

Parameters are fixed

and

Data is random

Bayesian statistics takes a different view.

In Bayesian statistics:

Parameters are uncertain

and data helps us reduce that uncertainty.

Rather than estimating:

One Best Value

Bayesian methods estimate:

A Probability Distribution

for the parameter.

Why Bayesian Statistics?

Suppose we want to estimate:

Probability of Sale

for a new inventory program.

A frequentist model might estimate:

0.65

A Bayesian model might estimate:

Most likely: 0.65
95% Probability Interval:
0.58 to 0.72

The Bayesian answer naturally expresses uncertainty.

The Core Idea

Bayesian statistics is simply:

Prior Beliefs
+
New Data
=
Updated Beliefs

This updating process is performed using Bayes’ Theorem.

Bayes’ Theorem

P(\theta|D)=\frac{P(D|\theta)P(\theta)}{P(D)}

where:

Understanding the Components

Prior

What we believe before seeing data.

Example:

Most diamonds sell
within 180 days.

This belief becomes a probability distribution.

Likelihood

Evidence from the observed data.

Example:

Actual sales records

Posterior

Updated belief after observing data.

This is the final result we care about.

Simple Example

Suppose we believe:

Probability of Sale
≈ 50%

before seeing data.

Prior:

0.50

Then we observe:

80 sales
20 failures

The posterior shifts toward:

0.80

because the data is strong.

Bayesian Coin Flip Example

Suppose:

Heads = Sale
Tails = No Sale

Prior:

50%

We observe:

8 Heads
2 Tails

The posterior updates toward:

80%

This is Bayesian learning.

The Beta Distribution

The most common prior for probabilities.

$$\theta\sim Beta(\alpha,\beta)$$

Examples:

Uniform Prior:

Beta(1,1)

Strong Belief Around 50%:

Beta(50,50)

Strong Belief Around 80%:

Beta(80,20)

Bayesian Updating

Prior:

Beta(1,1)

Observe:

80 Sales
20 Non-Sales

Posterior:

Beta(81,21)

Notice:

Prior + Data
=
Posterior

Visualizing the Posterior

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import beta
x = np.linspace(0, 1, 1000)
plt.plot(
    x,
    beta.pdf(x, 81, 21)
)
plt.title(
    "Posterior Distribution"
)
plt.xlabel(
    "Probability of Sale"
)
plt.ylabel(
    "Density"
)
plt.show()

Posterior Mean

For a Beta distribution:

E(\theta)=\frac{\alpha}{\alpha+\beta}

Example:

Posterior:

Beta(81,21)

Mean:

81 / (81 + 21)

Output:

0.794

Interpretation:

Estimated sale probability
≈ 79.4%

Credible Intervals

Bayesian equivalent of confidence intervals.

Calculate:

from scipy.stats import beta
lower = beta.ppf(
    0.025,
    81,
    21
)
upper = beta.ppf(
    0.975,
    81,
    21
)
print(lower, upper)

Example:

0.70 0.87

Interpretation:

95% probability
that the true parameter
lies inside the interval

This interpretation is often easier than frequentist confidence intervals.

Bayesian Linear Regression

Instead of estimating:

One slope

Bayesian regression estimates:

A distribution
for the slope

Model:

y=\beta_0+\beta_1x+\varepsilon

But now:

\beta_1\sim N(0,10^2)

The slope itself is uncertain.

Why This Matters

Frequentist result:

Slope = 2.1

Bayesian result:

Slope
Mean = 2.1
95% Credible Interval
1.6 to 2.8

Much richer information.

Bayesian Modeling with PyMC

Install:

pip install pymc

Import:

import pymc as pm

Simple Bayesian Model

import pymc as pm
with pm.Model() as model:
    theta = pm.Beta(
        "theta",
        alpha=1,
        beta=1
    )
    observations = pm.Bernoulli(
        "obs",
        p=theta,
        observed=[
            1,1,1,1,1,
            1,1,1,0,0
        ]
    )
    trace = pm.sample(
        2000,
        random_seed=42
    )

Posterior Summary

import arviz as az
az.summary(trace)

Output:

Posterior Distribution

az.plot_posterior(
    trace,
    var_names=["theta"]
)

This visualization is central to Bayesian analysis.

Healthcare Example

Question:

What is the probability
a patient is readmitted?

Data:

Readmitted
Not Readmitted

Use:

Beta-Binomial Model

Result:

Posterior Probability
of Readmission

Supply Chain Example

Question:

What is the probability
inventory sells
within 180 days?

Data:

Sold
Not Sold

Bayesian updating provides:

Probability Distribution

instead of a single estimate.

Bayesian Linear Regression in PyMC

with pm.Model() as model:
    beta0 = pm.Normal(
        "beta0",
        mu=0,
        sigma=10
    )
    beta1 = pm.Normal(
        "beta1",
        mu=0,
        sigma=10
    )
    sigma = pm.HalfNormal(
        "sigma",
        sigma=10
    )
    mu = (
        beta0
        + beta1 * x
    )
    y_obs = pm.Normal(
        "y_obs",
        mu=mu,
        sigma=sigma,
        observed=y
    )
    trace = pm.sample(
        2000
    )

Posterior Predictions

Generate future predictions.

with model:
    posterior_pred = (
        pm.sample_posterior_predictive(
            trace
        )
    )

This naturally includes uncertainty.

Advantages of Bayesian Methods

Direct Uncertainty Quantification

Provides full distributions.

Incorporates Prior Knowledge

Useful when data is limited.

Natural Probabilistic Interpretation

Example:

95% probability
parameter lies here

Handles Complex Models

Hierarchical models become straightforward.

Limitations

Computationally Intensive

Often requires:

MCMC

or

Variational Inference

Requires Prior Selection

Choice of prior matters.

More Complex

Than standard regression.

Typical Analyst Workflow

Step 1

Define prior.

Step 2

Specify likelihood.

Step 3

Fit model.

pm.sample()

Step 4

Inspect posterior.

az.summary()

Step 5

Visualize posterior.

az.plot_posterior()

Step 6

Generate predictions.

sample_posterior_predictive()

Practical Healthcare Exercise

Estimate:

Probability of Readmission

using:

• Age
• BMI
• Blood Pressure
• Prior Admissions

Questions:

• What is the posterior probability?
• What uncertainty remains?

Practical Supply Chain Exercise

Estimate:

Probability of Sale

using:

• Price
• Shape
• Color
• Clarity
• Customer

Questions:

• What inventory is most likely to sell?
• What uncertainty exists around predictions?

Lesson Summary

In this lesson we learned:

• Bayes’ Theorem
• Priors
• Likelihoods
• Posteriors
• Beta-Binomial Models
• Credible Intervals
• Bayesian Regression
• PyMC
• Posterior Prediction
• Healthcare Applications
• Supply Chain Applications

Bayesian Modeling is one of the most powerful frameworks in statistics because it treats uncertainty as a first-class citizen and continuously updates beliefs as new data arrives.

In the final lesson, we will study Causal Inference, the discipline of answering the most important question in analytics:

Did X actually cause Y?

rather than simply:

Are X and Y associated?