Introduction

In the previous lesson, we introduced weak convergence.

We learned that:

$$\mu_n \Rightarrow \mu$$

means:

$$\int f,d\mu_n \to \int f,d\mu$$

for every bounded continuous function:

$$f$$

Weak convergence provides a beautiful notion of convergence for measures.

However, a major question immediately appears:

How do we know a convergent subsequence exists at all?

For functions, compactness theorems often answer this question.

For example:

• Bolzano–Weierstrass
• Arzelà–Ascoli
• Banach–Alaoglu

In probability and measure theory, the analogous concept is tightness.

Tightness tells us whether a family of measures is prevented from “escaping to infinity.”

The central theorem connecting tightness and weak convergence is Prokhorov’s Theorem.

This theorem is one of the foundations of modern probability theory.

---

The Problem

Consider:

$$\mu_n=\delta_n$$

on:

$$\mathbb R$$

Each measure places all mass at:

$$n$$

As:

$$n\to\infty$$

the mass moves farther and farther away.

There is no sensible limiting probability measure on:

$$\mathbb R$$

The probability mass is escaping toward infinity.

Thus:

$$\delta_n$$

does not converge weakly.

---

What Went Wrong?

The measures fail because their mass is not staying inside any bounded region.

No compact set can capture most of the probability mass.

This observation motivates the definition of tightness.

---

Definition of Tightness

A family of probability measures:

$${\mu_n}$$

on a metric space:

$$X$$

is called tight if:

for every:

$$\varepsilon>0$$

there exists a compact set:

$$K\subseteq X$$

such that:

$$\mu_n(K)>1-\varepsilon$$

for every:

$$n$$

---

Intuition

Tightness means:

Almost all of the probability mass remains trapped inside a common compact set.

No significant amount of mass can escape to infinity.

---

Visual Interpretation

Imagine probability mass as sand.

Tightness means:

• the sand may move around
• the sand may reshape itself
• but almost all of it remains inside a fixed container

Without tightness, the sand can drift away indefinitely.

---

Example 1: A Tight Family

Suppose:

$$\mu_n=N(0,1)$$

for every:

$$n$$

All measures are identical.

Choose a sufficiently large interval:

$$[-M,M]$$

Then:

$$P(|X|>M)<\varepsilon$$

for every:

$$n$$

Therefore:

$${\mu_n}$$

is tight.

---

Example 2: Non-Tight Family

Consider:

$$\mu_n=\delta_n$$

Fix any compact set:

$$K$$

Since:

$$K$$

is bounded,

eventually:

$$n\notin K$$

Therefore:

$$\delta_n(K)=0$$

for large:

$$n$$

No compact set captures most of the mass.

Hence the family is not tight.

---

Why Tightness Matters

Weak convergence can occur only if mass does not escape.

Tightness is the condition that prevents this.

It plays the same role that boundedness plays in classical compactness theorems.

---

Relative Compactness

A family of measures is called relatively compact if every sequence contains a weakly convergent subsequence.

This is precisely what we want in many applications.

The question becomes:

What condition guarantees relative compactness?

The answer is Prokhorov’s Theorem.

---

Statement of Prokhorov’s Theorem

Let:

$$X$$

be a complete separable metric space.

Such spaces are called Polish spaces.

Examples include:

• $$\mathbb R$$
• $$\mathbb R^n$$
• Separable Hilbert spaces

Then:

A family of probability measures is tight if and only if it is relatively compact under weak convergence.

---

Why This Is Important

The theorem says:

$$\text{Tightness} \iff \text{Weak Sequential Compactness}$$

This is the probability-theory analogue of:

$$\text{Boundedness} \Rightarrow \text{Compactness}$$

in finite-dimensional geometry.

---

What Prokhorov Really Gives Us

Suppose:

$${\mu_n}$$

is tight.

Then every sequence contains a subsequence:

$$\mu_{n_k}$$

and a measure:

$$\mu$$

such that:

$$\mu_{n_k}\Rightarrow\mu$$

This is enormously useful.

It allows us to prove existence of limiting measures.

---

Example

Suppose:

$$X_n\sim N(0,\sigma_n^2)$$

with:

$$0\le\sigma_n\le1$$

The variances remain bounded.

The family is tight.

Therefore Prokhorov’s Theorem guarantees:

a weakly convergent subsequence exists.

Even without knowing the limit explicitly.

---

Why Analysts Love Compactness

Many proofs follow the same pattern.

Step 1

Show a family is tight.

Step 2

Apply Prokhorov.

Step 3

Extract a convergent subsequence.

Step 4

Identify the limit.

This strategy appears throughout modern analysis.

---

Tightness in Probability

Suppose:

$$X_n$$

is a sequence of random variables.

The distributions:

$$\mu_n$$

are tight if:

for every:

$$\varepsilon>0$$

there exists:

$$M>0$$

such that:

$$P(|X_n|>M)<\varepsilon$$

for all:

$$n$$

This form is often easier to verify.

---

Example

Suppose:

$$E[X_n^2]\le C$$

for all:

$$n$$

Using Markov’s inequality:

$$P(|X_n|>M)\le\frac{C}{M^2}$$

As:

$$M\to\infty$$

the right-hand side approaches zero.

Thus the family is tight.

This is one of the most frequently used tightness arguments.

---

Why Tightness Is a Compactness Condition

Think about Bolzano–Weierstrass.

A bounded sequence in:

$$\mathbb R^n$$

cannot escape to infinity.

Therefore a convergent subsequence exists.

Tightness plays exactly the same role for measures.

It prevents probability mass from escaping.

---

Connection to Weak Convergence

Weak convergence alone studies limits that already exist.

Prokhorov’s Theorem helps answer:

Does a limit exist?

Thus weak convergence and tightness work together.

---

Application to Stochastic Processes

Later in probability theory we will study:

• Brownian motion
• Markov processes
• Diffusions

Existence proofs frequently rely on:

1. Tightness.
2. Prokhorov’s Theorem.
3. Weak convergence.

This methodology underlies much of modern stochastic analysis.

---

Application to Bayesian Statistics

Suppose:

$$\Pi_n$$

are posterior distributions.

To prove convergence of posteriors, one often shows:

1. Tightness of:

$$\Pi_n$$

2. Existence of weakly convergent subsequences.
3. Identification of the limiting distribution.

Prokhorov’s Theorem is therefore fundamental in Bayesian asymptotics.

---

Application to Machine Learning

Many consistency proofs involve sequences of probability measures.

Examples include:

• Empirical distributions
• Posterior distributions
• Learned representations

Tightness often provides the compactness needed to establish convergence.

---

Philosophical Meaning

Tightness expresses a powerful principle:

Mass cannot disappear.

If probability mass remains concentrated in compact regions, then the sequence retains enough structure to produce a limit.

This idea appears repeatedly throughout mathematics.

---

Connection to Functional Analysis

You may notice a recurring pattern:

Classical Analysis

Probability Theory

Boundedness

Tightness

Compactness

Weak Compactness

Convergent Subsequence

Weakly Convergent Subsequence

Prokhorov’s Theorem is essentially a compactness theorem for probability measures.

---

Connection to Alain Connes

One of the central themes in modern mathematics is:

Understand infinite objects through compactness principles.

Tightness and Prokhorov’s Theorem are early examples of this philosophy.

Later:

• states on operator algebras,
• representations,
• spectral measures,

all require analogous compactness arguments.

The idea that controlling mass leads to existence of limits reappears throughout the analytical framework underlying Connes’ work.

---

Key Concepts Learned

By the end of this lesson you should understand:

• Tightness prevents probability mass from escaping to infinity.
• A family is tight if:

$$\mu_n(K)>1-\varepsilon$$

for some compact:

$$K$$

and every:

$$n$$

• Tightness is the probability analogue of boundedness.
• Prokhorov’s Theorem states:

$$\text{Tightness} \iff \text{Relative Compactness}$$

for probability measures on Polish spaces.

• Tightness guarantees weakly convergent subsequences.
• Prokhorov’s Theorem is fundamental in probability, statistics, and stochastic processes.

---

Looking Ahead

Measure Theory Lesson 33: Hausdorff Measure

So far, our measures have been adapted to ordinary geometric objects such as intervals, areas, and volumes. In the next lesson, we will encounter a revolutionary idea:

What if a set has a dimension that is not an integer?

Hausdorff measure provides a way to measure fractals, Cantor sets, and highly irregular geometric objects. This theory becomes one of the foundations of modern geometric measure theory and plays an important role in understanding the kinds of spaces that eventually motivate noncommutative geometry.