In the previous lesson, we learned that not every subset can be assigned a probability.

To avoid contradictions, we introduced a sigma algebra, which specifies the collection of sets that are allowed to receive probabilities.

Now we combine two ideas:

• A set of possible outcomes
• A sigma algebra of measurable events

The result is one of the most important objects in mathematics:

The Measurable Space

This object serves as the stage on which all of measure theory, probability theory, stochastic processes, and Bayesian nonparametrics are built.

---

Why We Need Measurable Spaces

Suppose we have a sample space:

$$\Omega={H,T}$$

This tells us what outcomes are possible.

However, it does not tell us which subsets are measurable.

To do that we need a sigma algebra:

$$\mathcal F={\emptyset,{H},{T},\Omega}$$

Together they form:

$$\left(\Omega,\mathcal F\right)$$

This pair is called a measurable space.

A measurable space tells us:

What outcomes exist and which collections of outcomes are measurable.

Before assigning probabilities, we must first construct this framework.

---

Formal Definition

A measurable space is a pair:

$$\left(X,\mathcal A\right)$$

where:

• $$X$$ is a set
• $$\mathcal A$$ is a sigma algebra on $$X$$

The elements of $$\mathcal A$$ are called measurable sets.

This is one of the simplest definitions in mathematics, yet almost all modern probability theory rests upon it.

---

Intuition

Think of:

$$X$$

as the universe.

Think of:

$$\mathcal A$$

as the collection of observable events.

Not everything in the universe is necessarily measurable.

Only the sets contained in:

$$\mathcal A$$

are allowed to participate in probability calculations.

---

Example 1: Coin Toss

Let:

$$X={H,T}$$

Choose:

$$\mathcal A={\emptyset,{H},{T},X}$$

Then:

$$\left(X,\mathcal A\right)$$

is a measurable space.

Every possible event is measurable.

This is the setting used in elementary probability.

---

Example 2: Rolling a Die

Sample space:

$$X={1,2,3,4,5,6}$$

Power set:

$$\mathcal A=2^X$$

The measurable space becomes:

$$\left(X,2^X\right)$$

Since every subset is measurable, we can assign probabilities to any event.

Examples:

Even numbers:

$${2,4,6}$$

Prime numbers:

$${2,3,5}$$

Numbers greater than four:

$${5,6}$$

All belong to the sigma algebra.

---

Example 3: Real Numbers

Now consider:

$$X=\mathbb R$$

This space contains infinitely many points.

Using the power set:

$$2^{\mathbb R}$$

creates serious problems because it contains non-measurable sets.

Instead we use:

$$\mathcal B(\mathbb R)$$

the Borel sigma algebra.

The measurable space becomes:

$$\left(\mathbb R,\mathcal B(\mathbb R)\right)$$

This is arguably the most important measurable space in probability theory.

---

Why Not Just Use the Whole Power Set?

For finite spaces:

$$2^X$$

works perfectly.

For infinite spaces, especially:

$$\mathbb R$$

the power set contains pathological objects.

The famous Vitali set cannot consistently receive a length.

If every subset were measurable, contradictions would arise.

Sigma algebras prevent this.

---

The Measurable Space Comes Before Probability

A common misconception is:

Probability comes first.

Actually the order is:

1. Set
2. Sigma algebra
3. Measurable space
4. Measure
5. Probability measure

Only after constructing:

$$\left(X,\mathcal A\right)$$

can we define a measure:

$$\mu:\mathcal A\rightarrow[0,\infty]$$

---

Visual Interpretation

Imagine a city.

The city itself is:

$$X$$

The streets that appear on your map are:

$$\mathcal A$$

The measurable space:

$$\left(X,\mathcal A\right)$$

is the city together with the roads you are allowed to travel.

A measure will later tell us the length of each road.

Probability will tell us how likely we are to visit each road.

---

Generating Sigma Algebras

Often we start with a collection of sets.

Example:

All open intervals:

$$(a,b)$$

We then construct the smallest sigma algebra containing them.

This sigma algebra is written:

$$\sigma(\mathcal C)$$

where:

$$\mathcal C$$

is the original collection.

For open intervals:

$$\sigma(\text{open intervals})=\mathcal B(\mathbb R)$$

This produces the Borel sigma algebra.

---

Generated Sigma Algebras

Suppose:

$$A={1,2}$$

inside:

$$X={1,2,3}$$

The sigma algebra generated by A is:

$$\sigma(A)={\emptyset,{1,2},{3},X}$$

Notice how closure properties force additional sets to appear.

Sigma algebras are often much larger than the collections that generate them.

---

Measurable Spaces in Statistics

Virtually every statistical model begins with a measurable space.

Normal distribution:

$$\left(\mathbb R,\mathcal B(\mathbb R)\right)$$

Poisson distribution:

$$\left(\mathbb N,2^{\mathbb N}\right)$$

Multivariate normal:

$$\left(\mathbb R^d,\mathcal B(\mathbb R^d)\right)$$

Without measurable spaces, probability distributions cannot even be defined.

---

Measurable Spaces and Random Variables

Soon we will define random variables.

A random variable is not merely a variable.

It is a function between measurable spaces.

Symbolically:

$$X:(\Omega,\mathcal F)\rightarrow(\mathbb R,\mathcal B(\mathbb R))$$

This perspective is one of the most important shifts in modern probability.

Random variables are mappings between measurable spaces.

---

Measurable Spaces and Bayesian Nonparametrics

In Bayesian statistics we place probability distributions on parameters.

In Bayesian nonparametrics we often place probability distributions on functions or probability measures themselves.

Examples:

Gaussian Process:

$$f\sim GP(m,k)$$

Dirichlet Process:

$$G\sim DP(\alpha,G_0)$$

These objects live in infinite-dimensional measurable spaces.

Understanding measurable spaces is the first step toward making sense of such models.

---

The Hierarchy So Far

We have now built the first three layers of measure theory:

Sets:

$$X$$

Sigma Algebras:

$$\mathcal A$$

Measurable Spaces:

$$(X,\mathcal A)$$

Next we will add measures:

$$\mu$$

which assign sizes to measurable sets.

This completes the transition from pure set theory into measure theory.

---

What You Should Know After This Lesson

You should now understand:

1. A measurable space is:

$$\left(X,\mathcal A\right)$$

2. A measurable space consists of:
   • a set
   • a sigma algebra
3. The measurable space defines which sets are measurable.
4. Measures can only be defined on measurable spaces.
5. Probability measures are special kinds of measures.
6. Random variables are mappings between measurable spaces.
7. Bayesian nonparametric models ultimately live in infinite-dimensional measurable spaces.

---

Mental Picture

Think of the progression as:

Set

$$\longrightarrow$$

Sigma Algebra

$$\longrightarrow$$

Measurable Space

$$\longrightarrow$$

Measure

$$\longrightarrow$$

Probability

Everything in modern probability theory is built by repeatedly extending this chain.

---

Preview of Lesson 4

Next we finally arrive at the central object of measure theory:

Measures

We will define exactly what a measure is, learn the three measure axioms, understand countable additivity, and see why length, area, volume, and probability are all examples of the same mathematical object.