Introduction

In the previous lesson, we learned that many real-world problems involve multiple variables interacting with one another.

Examples include:

Supply Chain

• Inventory
• Sales

Health

• Healthy cells
• Infected cells

Business

• Customers
• Advertising

Engineering

• Position
• Velocity

In each case, the variables influence each other.

The question now becomes:

How do we analyze these systems efficiently?

The answer is matrices.

This lesson introduces one of the most important concepts in all of applied mathematics:

Eigenvalues and Eigenvectors.

These ideas determine:

• Stability
• Long-term behavior
• Oscillations
• Growth
• Decay

and eventually become critical in:

• State-space models
• Kalman filters
• Stochastic differential equations
• Machine learning
• Bayesian dynamic models

---

Linear Systems

Consider the system

$$
\frac{dx}{dt}=2x+y
$$

$$
\frac{dy}{dt}=x+2y
$$

Notice that every term is linear.

No squares.

No products.

No logarithms.

This is called a linear system.

---

Matrix Representation

Define the state vector

$$
\mathbf{X}=\begin{bmatrix}
x\\
y
\end{bmatrix}
$$

Then the system becomes

$$
\frac{d\mathbf{X}}{dt}=\begin{bmatrix}
2 & 1\\
1 & 2
\end{bmatrix}
\mathbf{X}
$$

We often write this as

$$
\frac{d\mathbf{X}}{dt}=A\mathbf{X}
$$

where

$$
A=\begin{bmatrix}
2 & 1\\
1 & 2
\end{bmatrix}
$$

The matrix completely describes the system.

---

Why Matrices Matter

The future behavior of the system depends entirely on the matrix.

Everything we want to know is hidden inside:

$$
A
$$

The key is extracting that information.

This leads to eigenvalues.

---

Supply Chain Example

Suppose:

$$
I(t)
$$

= Inventory

$$
S(t)
$$

= Sales

A simplified system could be

$$
\frac{dI}{dt}=-0.3I-0.2S
$$

$$
\frac{dS}{dt}=0.4I-0.5S
$$

Matrix form:

$$
\frac{d}{dt}
\begin{bmatrix}
I\\
S
\end{bmatrix}=\begin{bmatrix}
-0.3 & -0.2\\
0.4 & -0.5
\end{bmatrix}
\begin{bmatrix}
I\\
S
\end{bmatrix}
$$

This model captures interaction between inventory and sales.

---

Health Example

Suppose:

$$
H(t)
$$

= Healthy cells

$$
I(t)
$$

= Infected cells

A simple linearized model might be

$$
\frac{dH}{dt}=-0.1I
$$

$$
\frac{dI}{dt}=0.1I-0.2H
$$

Matrix form:

$$
\frac{d}{dt}
\begin{bmatrix}
H\\
I
\end{bmatrix}=\begin{bmatrix}
0 & -0.1\\
-0.2 & 0.1
\end{bmatrix}
\begin{bmatrix}
H\\
I
\end{bmatrix}
$$

Understanding the matrix tells us whether infection grows or dies out.

---

Business Example

Suppose:

$$
A(t)
$$

= Advertising effectiveness

$$
C(t)
$$

= Customers

A model could be

$$
\frac{dA}{dt}=-0.4A
$$

$$
\frac{dC}{dt}=0.2A-0.1C
$$

Again we obtain a matrix system.

---

Engineering Example

Suppose:

$$
x(t)
$$

= Position

$$
v(t)
$$

= Velocity

Then

$$
\frac{dx}{dt}=v
$$

$$
\frac{dv}{dt}=-2x
$$

can be written as

$$
\frac{d}{dt}
\begin{bmatrix}
x\\
v
\end{bmatrix}=\begin{bmatrix}
0 & 1\\
-2 & 0
\end{bmatrix}
\begin{bmatrix}
x\\
v
\end{bmatrix}
$$

---

What Is an Eigenvalue?

Suppose a matrix acts on a vector.

Normally:

• the length changes
• the direction changes

However, some special vectors only change in size.

Their direction remains unchanged.

These are called eigenvectors.

The amount of stretching is called the eigenvalue.

Mathematically:

$$
A\mathbf{v}=\lambda\mathbf{v}
$$

where

$$
\mathbf{v}
$$

is an eigenvector and

$$
\lambda
$$

is the corresponding eigenvalue.

---

Why Are Eigenvalues Important?

Eigenvalues determine:

• Growth
• Decay
• Stability
• Oscillation

Without solving the differential equation.

---

Example Matrix

Consider

$$
A=
\begin{bmatrix}
2 & 0\\
0 & 3
\end{bmatrix}
$$

The eigenvalues are

$$
2
$$

and

$$
3
$$

The solution contains

$$
e^{2t}
$$

and

$$
e^{3t}
$$

Since both are positive:

The system grows exponentially.

---

Stability Rule

For

$$
\frac{d\mathbf{X}}{dt}=A\mathbf{X}
$$

the eigenvalues determine stability.

---

Case 1: All Eigenvalues Negative

Example:

$$
\lambda_1=-1
$$

$$
\lambda_2=-3
$$

Solutions contain

$$
e^{-t}
$$

and

$$
e^{-3t}
$$

Everything approaches zero.

Stable.

---

Supply Chain Interpretation

Inventory fluctuations disappear.

Sales stabilize.

The system settles down.

---

Health Interpretation

Infection dies out.

Patient recovers.

---

Business Interpretation

Customer numbers stabilize.

The market reaches equilibrium.

---

Case 2: At Least One Positive Eigenvalue

Example:

$$
\lambda_1=2
$$

$$
\lambda_2=-1
$$

The positive eigenvalue dominates.

The solution eventually explodes.

Unstable.

---

Supply Chain Interpretation

Inventory grows uncontrollably.

Excess stock accumulates.

---

Health Interpretation

Infection spreads rapidly.

Outbreak occurs.

---

Business Interpretation

Growth accelerates.

Customer base expands rapidly.

---

Case 3: Complex Eigenvalues

Example:

$$
\lambda=-1 \pm 2i
$$

Solutions oscillate.

Because the real part is negative:

Oscillations shrink over time.

---

Engineering Interpretation

A spring vibrates and gradually stops.

---

Supply Chain Interpretation

Inventory oscillates around target levels before stabilizing.

---

Health Interpretation

Biological markers fluctuate before reaching equilibrium.

---

Equilibrium of a Linear System

For

$$
\frac{d\mathbf{X}}{dt}=A\mathbf{X}
$$

equilibrium occurs when

$$
A\mathbf{X}=0
$$

The equilibrium is often

$$
\mathbf{X}=\mathbf{0}
$$

although more complicated systems may have multiple equilibria.

---

Why Statisticians Care About Eigenvalues

Eigenvalues appear everywhere in statistics.

Examples include:

Principal Component Analysis

Largest eigenvalues determine important directions.

Covariance Matrices

Eigenvalues measure variability.

Dynamic Models

Eigenvalues determine stability.

State-Space Models

Eigenvalues govern long-term behavior.

Kalman Filters

Eigenvalues affect convergence.

Stochastic Differential Equations

Eigenvalues determine whether randomness grows or disappears.

---

The Matrix Exponential

The solution of

$$
\frac{d\mathbf{X}}{dt}=A\mathbf{X}
$$

is

$$
\mathbf{X}(t)=e^{At}
\mathbf{X}(0)
$$

This is the matrix equivalent of

$$
y(t)=Ce^{rt}
$$

from ordinary differential equations.

Everything now depends on the matrix exponential.

And eigenvalues determine its behavior.

---

The Big Idea

For systems of differential equations:

The matrix contains the entire story.

Eigenvalues reveal:

• Stability
• Growth
• Decay
• Oscillation

without requiring us to solve the system directly.

This is one of the most powerful ideas in applied mathematics.

---

Looking Ahead

In Lesson 8 we will study:

Phase Planes and Trajectories

We will learn how to visualize systems of differential equations.

Topics include:

• Vector fields
• Direction fields
• Trajectories
• Attractors
• Spiral points
• Saddle points

This is where differential equations start to become geometric.

---

Key Takeaways

• Linear systems can be written as

$$
\frac{d\mathbf{X}}{dt}=A\mathbf{X}
$$

• Matrices describe interactions between variables.
• Eigenvalues determine long-term behavior.
• Negative eigenvalues imply stability.
• Positive eigenvalues imply instability.
• Complex eigenvalues create oscillations.
• Eigenvalues play a central role in statistics, health modeling, supply chains, engineering, and stochastic differential equations.