Solving Yule-Walker Equations to Derive ACF Values for an AR(2) Model

Q: Why do we assume $ E[Y_t] = 0 $ when deriving the Yule-Walker equations?

We assume $ E[Y_t] = 0 $ for simplicity. If $ E[Y_t] = \mu \neq 0 $, we can work with the mean-subtracted series $ Z_t = Y_t - \mu $. The AR model for $ Y_t $ becomes $ Y_t - \mu = \phi_1 (Y_{t-1} - \mu) + \phi_2 (Y_{t-2} - \mu) + \epsilon_t $, which is equivalent to an AR model for $ Z_t $. The autocorrelations are unaffected by subtracting the mean, so the Yule-Walker equations apply directly to the $ Z_t $ series and thus to the original $ Y_t $ series' ACF.

Q: What are the stationarity conditions for an AR(2) model?

An AR(2) process $ Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t $ is stationary if and only if the roots of its characteristic equation $ 1 - \phi_1 z - \phi_2 z^2 = 0 $ lie outside the unit circle. Equivalently, the coefficients must satisfy three conditions: $ \phi_1 + \phi_2 < 1 $, $ \phi_2 - \phi_1 < 1 $, and $ |\phi_2| < 1 $.

Q: How do these equations change for an AR(p) model?

For an AR(p) model, $ Y_t = \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \epsilon_t $, the Yule-Walker equations form a system of $ p $ linear equations for the first $ p $ autocorrelation coefficients: $ \rho(k) = \sum_{i=1}^p \phi_i \rho(k-i) $ for $ k=1, \dots, p $. These can be compactly written in matrix form as $ \rho_p = \Gamma_p \phi_p $, where $ \rho_p = [\rho(1), \dots, \rho(p)]^T $ and $ \Gamma_p $ is a symmetric Toeplitz matrix with $ (i,j) $-th entry $ \rho(|i-j|) $.

Q: Why is $ E[\epsilon_t Y_{t-k}] = 0 $ for $ k \ge 1 $?

This is a fundamental property of AR models, asserting that the white noise innovation $ \epsilon_t $ is uncorrelated with past values of the process. $ \epsilon_t $ represents the unpredictable part of $ Y_t $ given its past values. Since $ Y_{t-k} $ for $ k \ge 1 $ depends only on $ \epsilon_{t-k}, \epsilon_{t-k-1}, \dots $ (and past $ Y $s, which are themselves functions of past $ \epsilon $s), it is independent of (and therefore uncorrelated with) the current innovation $ \epsilon_t $. This uncorrelation is crucial for simplifying the expectation terms in the Yule-Walker derivation.

Master solving Yule-Walker equations for an AR(2) model to derive ACF values. Rigorous theory, intuitive insights, and common pitfalls for BSc students.

The Formal Theorem

For a stationary Autoregressive model of order 2, denoted AR(2), given by

Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t

, where

\epsilon_t

is white noise with

E[\epsilon_t] = 0

and

Var(\epsilon_t) = \sigma_{\epsilon}^2

, the Yule-Walker equations provide a relationship between the autoregressive coefficients

\phi_1, \phi_2

and the autocorrelation function (ACF) values

\rho(k)

. Specifically, for lags

k=1

and

k=2

, the equations are:

\begin{aligned} \rho(1) &= \phi_1 \rho(0) + \phi_2 \rho(1) \\ \rho(2) &= \phi_1 \rho(1) + \phi_2 \rho(0) \end{aligned}

Given that

\rho(0) = 1

for any stationary process, this system simplifies to:

\begin{aligned} \rho(1) &= \phi_1 + \phi_2 \rho(1) \quad & (1) \\ \rho(2) &= \phi_1 \rho(1) + \phi_2 \quad & (2) \end{aligned}

Solving equation (1) for

\rho(1)

yields:

\rho(1) (1 - \phi_2) = \phi_1 \implies \rho(1) = \frac{\phi_1}{1 - \phi_2}

Substituting this expression for

\rho(1)

into equation (2) gives

\rho(2)

\rho(2) = \phi_1 \left( \frac{\phi_1}{1 - \phi_2} \right) + \phi_2 = \frac{\phi_1^2}{1 - \phi_2} + \phi_2

Thus, the first two autocorrelation coefficients for a stationary AR(2) process are:

\begin{aligned} \rho(1) &= \frac{\phi_1}{1 - \phi_2} \\ \rho(2) &= \frac{\phi_1^2 + \phi_2(1 - \phi_2)}{1 - \phi_2} = \frac{\phi_1^2 + \phi_2 - \phi_2^2}{1 - \phi_2} \end{aligned}

For

k \ge 2

, the higher-order autocorrelations follow the general recursive relation:

\rho(k) = \phi_1 \rho(k-1) + \phi_2 \rho(k-2)

Analytical Intuition.

Imagine a majestic celestial dance, where a planet's current position,

Y_t

, is influenced by its positions in the immediate past:

Y_{t-1}

and

Y_{t-2}

. The Yule-Walker equations are the ancient cosmic laws governing this celestial ballet. They are not merely observations but the very DNA of the system, revealing how the planet's past movements (encoded by the autoregressive coefficients

\phi_1

and

\phi_2

) dictate its current and future trajectory. We're decoding the universe's internal rhythm.

\rho(1)

quantifies how much today's position remembers yesterday's, a direct gravitational tug, while

\rho(2)

unveils the lingering influence from two periods ago, a more subtle, yet powerful, orbital resonance. Solving these equations is akin to calibrating an orrery, transforming observed astronomical patterns into precise constants. It allows us to predict the planet's winding course with mathematical certainty, uncovering the hidden, rhythmic pulse that drives its majestic motion.

CAUTION

Institutional Warning.

Students often confuse the Yule-Walker equations with the Durbin-Levinson algorithm. While related, the Yule-Walker equations directly relate AR coefficients to ACF for a fixed order, whereas Durbin-Levinson recursively computes AR parameters and prediction errors for increasing orders based on known autocorrelations.

Institutional Deep Dive.

The Yule-Walker equations serve as the foundational bridge connecting the theoretical parameters of an autoregressive (AR) model to its observable autocorrelation structure. For an AR(2) process, defined by

Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t

, where

\epsilon_t

is white noise with

E[\epsilon_t] = 0

and

Var(\epsilon_t) = \sigma_{\epsilon}^2

, and

Y_t

is a stationary process with

E[Y_t] = 0

(which can always be assumed by working with deviations from the mean), the essence of the Yule-Walker equations lies in expressing the autocovariances (and thus autocorrelations) in terms of the model coefficients

\phi_1

and

\phi_2

and the noise variance

\sigma_{\epsilon}^2

**Core Logic** By multiplying the AR(2) equation by

Y_{t-k}

and taking expectations, we obtain a system of linear equations. For a stationary process,

Cov(Y_t, Y_{t-k}) = \gamma(k)

and

Var(Y_t) = \gamma(0)

. The crucial steps involve utilizing the property that

\epsilon_t

is uncorrelated with past values of the process

Y_{t-k}

for

k \ge 1

, i.e.,

E[\epsilon_t Y_{t-k}] = 0

. However,

E[\epsilon_t Y_t] = \sigma_{\epsilon}^2

because

Y_t

directly depends on

\epsilon_t

For

k=1

, multiplying the AR(2) equation by

Y_{t-1}

and taking expectations yields:

E[Y_t Y_{t-1}] = E[(\phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t) Y_{t-1}]

\gamma(1) = \phi_1 \gamma(0) + \phi_2 \gamma(1) + E[\epsilon_t Y_{t-1}]

Since

E[\epsilon_t Y_{t-1}] = 0

, this simplifies to

\gamma(1) = \phi_1 \gamma(0) + \phi_2 \gamma(1)

For

k=2

, multiplying the AR(2) equation by

Y_{t-2}

and taking expectations yields:

E[Y_t Y_{t-2}] = E[(\phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t) Y_{t-2}]

\gamma(2) = \phi_1 \gamma(1) + \phi_2 \gamma(0) + E[\epsilon_t Y_{t-2}]

Since

E[\epsilon_t Y_{t-2}] = 0

, this simplifies to

\gamma(2) = \phi_1 \gamma(1) + \phi_2 \gamma(0)

Dividing both equations by

\gamma(0)

(the variance of the process), we transform the autocovariances

\gamma(k)

into autocorrelations

\rho(k) = \gamma(k) / \gamma(0)

. Given that

\rho(0) = \gamma(0) / \gamma(0) = 1

, we arrive at the Yule-Walker equations for an AR(2) model in terms of autocorrelations:

\rho(1) = \phi_1 + \phi_2 \rho(1)

\rho(2) = \phi_1 \rho(1) + \phi_2

This is a straightforward linear system in

\rho(1)

and

\rho(2)

that can be solved algebraically, yielding unique solutions provided the stationarity conditions for the AR(2) model are met (specifically,

1 - \phi_2 \neq 0

which is implied by stationarity).

**Geometric Mechanics** From a geometric perspective, the Yule-Walker equations can be understood through the lens of orthogonal projections in a Hilbert space of random variables. Each

Y_t

can be thought of as a vector. The AR(2) model essentially states that

Y_t

is the best linear predictor of itself based on

Y_{t-1}

and

Y_{t-2}

, plus an orthogonal error term

\epsilon_t

. This orthogonality means

\epsilon_t

is uncorrelated with

Y_{t-1}

and

Y_{t-2}

. The act of multiplying the AR(2) equation by

Y_{t-k}

and taking expectations is analogous to computing an inner product in this Hilbert space. The vanishing of

E[\epsilon_t Y_{t-k}]

for

k \ge 1

is precisely the orthogonality condition: the innovations

\epsilon_t

are uncorrelated with all past observations. The Yule-Walker equations thus mathematically formalize this projection and orthogonality, expressing the autocorrelations as direct consequences of the model's intrinsic predictive structure. Solving them reveals the specific pattern of how the process's memory decays over time, akin to uncovering its unique spectral fingerprint.

**Institutional Pitfalls** A common pitfall for students is to forget the crucial role of stationarity. The derivation of the Yule-Walker equations critically relies on the assumption that

Y_t

is weakly stationary, which implies constant mean, constant variance, and autocovariance depending only on the lag. Without stationarity,

\gamma(k)

and

\rho(k)

are not well-defined or constant, and the equations do not hold in this simplified form. Another error is incorrectly assuming

E[\epsilon_t Y_{t-k}] = 0

for all

k

, including

k=0

. While

\epsilon_t

is uncorrelated with past

Y_s

(for

s < t

), it is inherently correlated with

Y_t

itself (as

Y_t

directly depends on

\epsilon_t

), leading to

E[Y_t \epsilon_t] = \sigma_{\epsilon}^2

, not zero. Furthermore, students sometimes incorrectly apply the general recursive relation

\rho(k) = \phi_1 \rho(k-1) + \phi_2 \rho(k-2)

for

k=1, 2

without first solving the system, which would lead to circular definitions. The first few equations must be solved as a system to establish the initial conditions for the recursive decay. Finally, careful algebraic manipulation is paramount, as small errors in solving for

\rho(1)

and

\rho(2)

can propagate and invalidate subsequent calculations of higher-order autocorrelations.

Academic Inquiries.

Why do we assume $E[Y_t] = 0$ when deriving the Yule-Walker equations?

We assume $E[Y_t] = 0$ for simplicity. If $E[Y_t] = \mu \neq 0$ , we can work with the mean-subtracted series $Z_t = Y_t - \mu$ . The AR model for $Y_t$ becomes $Y_t - \mu = \phi_1 (Y_{t-1} - \mu) + \phi_2 (Y_{t-2} - \mu) + \epsilon_t$ , which is equivalent to an AR model for $Z_t$ . The autocorrelations are unaffected by subtracting the mean, so the Yule-Walker equations apply directly to the $Z_t$ series and thus to the original $Y_t$ series' ACF.

What are the stationarity conditions for an AR(2) model?

An AR(2) process $Y_t = \phi_1 Y_{t-1} + \phi_2 Y_{t-2} + \epsilon_t$ is stationary if and only if the roots of its characteristic equation $1 - \phi_1 z - \phi_2 z^2 = 0$ lie outside the unit circle. Equivalently, the coefficients must satisfy three conditions: $\phi_1 + \phi_2 < 1$ , $\phi_2 - \phi_1 < 1$ , and $|\phi_2| < 1$ .

How do these equations change for an AR(p) model?

For an AR(p) model, $Y_t = \phi_1 Y_{t-1} + \dots + \phi_p Y_{t-p} + \epsilon_t$ , the Yule-Walker equations form a system of $p$ linear equations for the first $p$ autocorrelation coefficients: $\rho(k) = \sum_{i=1}^p \phi_i \rho(k-i)$ for $k=1, \dots, p$ . These can be compactly written in matrix form as $\rho_p = \Gamma_p \phi_p$ , where $\rho_p = [\rho(1), \dots, \rho(p)]^T$ and $\Gamma_p$ is a symmetric Toeplitz matrix with $(i,j)$ -th entry $\rho(|i-j|)$ .

Why is $E[\epsilon_t Y_{t-k}] = 0$ for $k \ge 1$ ?

This is a fundamental property of AR models, asserting that the white noise innovation $\epsilon_t$ is uncorrelated with past values of the process. $\epsilon_t$ represents the unpredictable part of $Y_t$ given its past values. Since $Y_{t-k}$ for $k \ge 1$ depends only on $\epsilon_{t-k}, \epsilon_{t-k-1}, \dots$ (and past $Y$ s, which are themselves functions of past $\epsilon$ s), it is independent of (and therefore uncorrelated with) the current innovation $\epsilon_t$ . This uncorrelation is crucial for simplifying the expectation terms in the Yule-Walker derivation.

Standardized References.

Definitive Institutional SourceBrockwell, P. J., & Davis, R. A. (2016). Introduction to Time Series and Forecasting (3rd ed.). Springer.

Intermediate

Proof that Autocovariance Depends Only on Lag for Weakly Stationary Processes

Explore the rigorous proof that autocovariance for weakly stationary processes depends only on lag, understanding its deep implications for time series analysis.

Foundational

Derivation of the Autocorrelation Function (ACF) for a White Noise Process

Derive the Autocorrelation Function (ACF) for white noise. Understand its theoretical properties and intuitive meaning in Time Series Analysis.

Intermediate

Proof of the Stationarity Condition for an AR(1) Process (|φ| < 1)

Unravel the stationarity condition for AR(1) processes. Rigorous proof, cinematic intuition, and essential insights for time series analysis.

Intermediate

Proof of the Invertibility Condition for an MA(1) Process (|θ| < 1)

Exploring the cinematic intuition of Proof of the Invertibility Condition for an MA(1) Process (|θ| < 1).

Institutional Citation

Reference this proof in your academic research or publications.

NICEFA Visual Mathematics. (2026). Solving Yule-Walker Equations to Derive ACF Values for an AR(2) Model: Visual Proof & Intuition. Retrieved from https://www.nicefa.org/library/time-series-analysis/solving-yule-walker-equations-to-derive-acf-values-for-an-ar-2--model

Dominate the Logic.

"Abstract theory is just a movement we haven't seen yet."

Master the Proof Early Access

The Formal Theorem

Analytical Intuition.

Institutional Warning.

Institutional Deep Dive.

Academic Inquiries.

Why do we assume E[Yt]=0 E[Y_t] = 0 E[Yt​]=0 when deriving the Yule-Walker equations?

What are the stationarity conditions for an AR(2) model?

How do these equations change for an AR(p) model?

Why is E[ϵtYt−k]=0 E[\epsilon_t Y_{t-k}] = 0 E[ϵt​Yt−k​]=0 for k≥1 k \ge 1 k≥1?

Standardized References.

Related Proofs Cluster.

Proof that Autocovariance Depends Only on Lag for Weakly Stationary Processes

Derivation of the Autocorrelation Function (ACF) for a White Noise Process

Proof of the Stationarity Condition for an AR(1) Process (|φ| < 1)

Proof of the Invertibility Condition for an MA(1) Process (|θ| < 1)

Institutional Citation

Dominate the Logic.

Why do we assume $E[Y_t] = 0$ when deriving the Yule-Walker equations?

Why is $E[\epsilon_t Y_{t-k}] = 0$ for $k \ge 1$ ?