NICEFA Logo
nicefa.
Library
←Module

The Gauss-Markov Theorem: Proof that OLS is the Best Linear Unbiased Estimator (BLUE)

Master the Gauss-Markov Theorem: Understand why OLS is the Best Linear Unbiased Estimator (BLUE) under key assumptions for robust statistical inference.

Visualizing...

Our institutional research engineers are currently mapping the formal proof for The Gauss-Markov Theorem: Proof that OLS is the Best Linear Unbiased Estimator (BLUE).

Apply for Institutional Early Access →

The Formal Theorem

Consider the linear regression model: y=Xβ+ϵ y = X\beta + \epsilon \\ where y y is an n×1 n \times 1 vector of observations, X X is an n×k n \times k matrix of regressors (non-stochastic, with full column rank k k ), β \beta is a k×1 k \times 1 vector of unknown parameters, and ϵ \epsilon is an n×1 n \times 1 vector of random error terms.\\ \\ \textbf{Gauss-Markov Assumptions:}\\ 1. \textbf{Linearity:} The model is linear in parameters: y=Xβ+ϵ y = X\beta + \epsilon .\\ 2. \textbf{Strict Exogeneity:} E[ϵ∣X]=0 E[\epsilon | X] = 0 . This implies E[ϵi]=0 E[\epsilon_i] = 0 for all i i .\\ 3. \textbf{No Perfect Multicollinearity:} rank(X)=k \text{rank}(X) = k . The design matrix X X has full column rank.\\ 4. \textbf{Homoscedasticity:} The error terms have constant variance: Var(ϵi∣X)=σ2 \text{Var}(\epsilon_i | X) = \sigma^2 for all i i .\\ 5. \textbf{No Autocorrelation:} The error terms are uncorrelated: Cov(ϵi,ϵj∣X)=0 \text{Cov}(\epsilon_i, \epsilon_j | X) = 0 for i≠j i \neq j .\\ \\ \textbf{Theorem Statement:}\\ Under the Gauss-Markov assumptions (1-5), the Ordinary Least Squares (OLS) estimator β^OLS \hat{\beta}_{OLS} of β \beta is the Best Linear Unbiased Estimator (BLUE). This means that among all estimators β~ \tilde{\beta} that are linear functions of y y and unbiased for β \beta , β^OLS \hat{\beta}_{OLS} has the smallest variance-covariance matrix in the sense that Var(β~)−Var(β^OLS) \text{Var}(\tilde{\beta}) - \text{Var}(\hat{\beta}_{OLS}) is a positive semi-definite matrix.\\ \\ The OLS estimator is given by:\
β^OLS=(XTX)−1XTy \hat{\beta}_{OLS} = (X^T X)^{-1} X^T y
\ The variance-covariance matrix of the OLS estimator is:\
Var(β^OLS)=σ2(XTX)−1 \text{Var}(\hat{\beta}_{OLS}) = \sigma^2 (X^T X)^{-1}

Analytical Intuition.

Imagine yourself as a master architect, tasked with designing the most stable and precise structure possible from a scattered set of observations y y . You have a blueprint X X and a hidden perfect design β \beta . Your goal is to infer β \beta using a 'straight-edge' rule (linear estimator) that, on average, hits the true target (unbiased). The OLS method β^OLS=(XTX)−1XTy \hat{\beta}_{OLS} = (X^T X)^{-1} X^T y is your chosen craftsman. The Gauss-Markov theorem is the ultimate quality assurance stamp: under ideal workshop conditions—such as perfectly uniform materials (homoscedasticity), no hidden interactions between measurements (no autocorrelation), and an inherently true design (strict exogeneity)—OLS is not just good, it's the absolute 'Best' linear, unbiased craftsman, producing estimates with the minimal possible wobble or imprecision (variance).
CAUTION

Institutional Warning.

Students often misinterpret 'Best' to mean OLS is the optimal estimator under all conditions. It strictly implies minimum variance \textit{only} among linear and unbiased estimators. They also frequently confuse the Gauss-Markov assumptions with those required for asymptotic properties like consistency or normality.

Academic Inquiries.

01

Does the Gauss-Markov Theorem require the error terms ϵ \epsilon to be normally distributed?

No, the Gauss-Markov Theorem does not require the errors ϵ \epsilon to be normally distributed. It only relies on the first and second moments of the errors (mean and variance/covariance). Normality is often assumed for hypothesis testing and confidence intervals, or for OLS to be the Maximum Likelihood Estimator (MLE), but it is not necessary for OLS to be BLUE.

02

If homoscedasticity or no autocorrelation is violated, is OLS still unbiased?

Yes, OLS remains unbiased even in the presence of heteroscedasticity or autocorrelation, provided the strict exogeneity assumption E[ϵ∣X]=0 E[\epsilon | X] = 0 holds. However, its variance-covariance matrix will be incorrectly estimated by the standard formula, and OLS will no longer be the 'Best' (most efficient) linear unbiased estimator. Other methods like Weighted Least Squares (WLS) or Generalized Least Squares (GLS) would be more efficient.

03

What does it mean for an estimator to be 'linear' in the context of OLS?

An estimator β~ \tilde{\beta} is 'linear' if it can be expressed as a linear function of the observed dependent variable y y . For OLS, β^OLS=(XTX)−1XTy \hat{\beta}_{OLS} = (X^T X)^{-1} X^T y . If we let C=(XTX)−1XT C = (X^T X)^{-1} X^T , then β^OLS=Cy \hat{\beta}_{OLS} = Cy , which clearly shows its linearity in y y . This linearity simplifies analysis and makes the estimator analytically tractable.

04

Why is the positive semi-definite condition for Var(β~)−Var(β^OLS) \text{Var}(\tilde{\beta}) - \text{Var}(\hat{\beta}_{OLS}) important?

The condition that Var(β~)−Var(β^OLS) \text{Var}(\tilde{\beta}) - \text{Var}(\hat{\beta}_{OLS}) is a positive semi-definite matrix means that for any non-zero vector a a , aT(Var(β~)−Var(β^OLS))a≥0 a^T (\text{Var}(\tilde{\beta}) - \text{Var}(\hat{\beta}_{OLS})) a \ge 0 . This implies that Var(aTβ~)≥Var(aTβ^OLS) \text{Var}(a^T \tilde{\beta}) \ge \text{Var}(a^T \hat{\beta}_{OLS}) for any linear combination of the elements of β~ \tilde{\beta} . In simpler terms, it means that the variance of any single coefficient estimate, or any linear combination of coefficients, from β^OLS \hat{\beta}_{OLS} will be less than or equal to that from any other linear unbiased estimator β~ \tilde{\beta} .

Standardized References.

  • Definitive Institutional SourceWooldridge, J. M. (2010). Econometric Analysis of Cross Section and Panel Data. MIT press.

Related Proofs Cluster.

Advanced

The Matrix Formulation of the General Linear Model: Y = Xβ + ϵ and its Fundamental Assumptions

Master the matrix formulation of the General Linear Model, \( Y = X\beta + \epsilon \), and its fundamental assumptions. Rigorous yet intuitive content for BSc Math/Stats students.

Foundational

Derivation of the Ordinary Least Squares (OLS) Estimator: β̂ = (X'X)⁻¹X'Y

Master the OLS estimator derivation: \( \hat{\beta} = (X'X)^{-1}X'Y \). Explore the geometric orthogonality, matrix calculus, and Gauss-Markov foundations.

Foundational

Proof of Unbiasedness of the OLS Estimator: E(β̂) = β

Master the rigorous proof of OLS estimator unbiasedness, \( E(\hat{\boldsymbol{\beta}}) = \boldsymbol{\beta} \). Understand critical assumptions, geometric intuition, and common pitfalls for robust linear modeling.

Foundational

Derivation of the Variance-Covariance Matrix of the OLS Estimator: Var(β̂) = σ²(X'X)⁻¹

A rigorous derivation of the Variance-Covariance matrix for the OLS estimator, exploring the geometric impact of data configuration on statistical precision.

Institutional Citation

Reference this proof in your academic research or publications.

NICEFA Visual Mathematics. (2026). The Gauss-Markov Theorem: Proof that OLS is the Best Linear Unbiased Estimator (BLUE): Visual Proof & Intuition. Retrieved from https://www.nicefa.org/library/general-linear-models-/the-gauss-markov-theorem--proof-that-ols-is-the-best-linear-unbiased-estimator--blue-

Dominate the Logic.

"Abstract theory is just a movement we haven't seen yet."

Subscribe for Full ProofsEarly Access