Theoretical Basis of Influence Diagnostics: Cook's Distance, DFFITS, and DFBETAS

Q: Can influence diagnostics be used in models with non-constant variance?

Standard diagnostics assume $ \text{Var}(\epsilon) = \sigma^2 I $. In heteroscedastic cases, weighted least squares or robust covariance estimators must be employed to avoid misinterpreting variance patterns as influence.

Q: What happens to DFBETAS when the design matrix is collinear?

Severe multicollinearity inflates the standard errors of $ \beta_j $, making the denominator of DFBETAS large and potentially masking the influence of specific observations.

Master influence diagnostics: Cook's Distance, DFFITS, and DFBETAS. Learn the geometric and theoretical basis for detecting influential data in linear models.

Visualizing...

Our institutional research engineers are currently mapping the formal proof for Theoretical Basis of Influence Diagnostics: Cook's Distance, DFFITS, and DFBETAS.

Apply for Institutional Early Access →

The Formal Theorem

Let the model be defined by

Y = X\beta + \epsilon

where

\epsilon \sim N(0, \sigma^2 I)

. Let

\hat{\beta}

be the OLS estimator and

\hat{\beta}_{(-i)}

be the estimator excluding the

i

-th observation. The influence measures are defined as follows:

\begin{aligned} D_i &= \frac{(\hat{\beta}_{(-i)} - \hat{\beta})^T (X^T X) (\hat{\beta}_{(-i)} - \hat{\beta})}{p s^2} = \frac{r_i^2}{p} \left( \frac{h_{ii}}{1 - h_{ii}} \right) \\ \text{DFFITS}_i &= \frac{\hat{y}_i - \hat{y}_{i(-i)}}{s_{(-i)} \sqrt{h_{ii}}} = r_i \sqrt{\frac{h_{ii}}{1 - h_{ii}}} \\ \text{DFBETAS}_{ij} &= \frac{\hat{\beta}_j - \hat{\beta}_{j(-i)}}{s_{(-i)} \sqrt{(X^T X)^{-1}_{jj}}} \end{aligned}

where

r_i

is the studentized residual,

h_{ii}

is the leverage, and

s^2

is the MSE.

Analytical Intuition.

Imagine you are constructing a bridge based on a set of coordinate data points

X

. Most points act as steady pillars, grounding the structure in statistical reality. However, a single rogue observation

(x_i, y_i)

acts like a hidden fault line; by shifting just a few millimeters, it can exert disproportionate leverage

h_{ii}

on the entire architectural design. Cook's Distance

D_i

is the seismic sensor measuring the total kinetic displacement of our regression parameters

\hat{\beta}

if that observation were suddenly removed. DFFITS looks specifically at the collapse of the predicted value

\hat{y}_i

, while DFBETAS tracks how individual structural components

\beta_j

buckle under the strain. We are not just fitting a line; we are identifying the 'opinionated' points that hold the model hostage. If

D_i

is high, the model's integrity is compromised, and the entire edifice risks falling over due to the influence of a singular, possibly erroneous, data input.

CAUTION

Institutional Warning.

Students often conflate 'leverage' with 'influence.' Leverage $h_{ii}$ is a function of $X$ alone and measures potential impact, whereas influence (like Cook's $D_i$ ) incorporates the response $Y$ . An observation can have high leverage but negligible influence if it aligns with the overall trend.

Academic Inquiries.

Why is the threshold for Cook's distance often set at 4/n?

The 4/n rule is a heuristic approximation suggesting that an observation is influential if its removal moves the parameter vector by more than the average individual contribution of the observations.

Can influence diagnostics be used in models with non-constant variance?

Standard diagnostics assume $\text{Var}(\epsilon) = \sigma^2 I$ . In heteroscedastic cases, weighted least squares or robust covariance estimators must be employed to avoid misinterpreting variance patterns as influence.

What happens to DFBETAS when the design matrix is collinear?

Severe multicollinearity inflates the standard errors of $\beta_j$ , making the denominator of DFBETAS large and potentially masking the influence of specific observations.

Standardized References.

Definitive Institutional SourceCook, R. D., & Weisberg, S. (1982). Residuals and Influence in Regression.

Advanced

The Matrix Formulation of the General Linear Model: Y = Xβ + ϵ and its Fundamental Assumptions

Master the matrix formulation of the General Linear Model, $ Y = X\beta + \epsilon $, and its fundamental assumptions. Rigorous yet intuitive content for BSc Math/Stats students.

Foundational

Derivation of the Ordinary Least Squares (OLS) Estimator: β̂ = (X'X)⁻¹X'Y

Master the OLS estimator derivation: $ \hat{\beta} = (X'X)^{-1}X'Y $. Explore the geometric orthogonality, matrix calculus, and Gauss-Markov foundations.

Foundational

Proof of Unbiasedness of the OLS Estimator: E(β̂) = β

Master the rigorous proof of OLS estimator unbiasedness, $ E(\hat{\boldsymbol{\beta}}) = \boldsymbol{\beta} $. Understand critical assumptions, geometric intuition, and common pitfalls for robust linear modeling.

Foundational

Derivation of the Variance-Covariance Matrix of the OLS Estimator: Var(β̂) = σ²(X'X)⁻¹

A rigorous derivation of the Variance-Covariance matrix for the OLS estimator, exploring the geometric impact of data configuration on statistical precision.

Institutional Citation

Reference this proof in your academic research or publications.

NICEFA Visual Mathematics. (2026). Theoretical Basis of Influence Diagnostics: Cook's Distance, DFFITS, and DFBETAS: Visual Proof & Intuition. Retrieved from https://www.nicefa.org/library/general-linear-models-/theoretical-basis-of-influence-diagnostics--cook-s-distance--dffits--and-dfbetas

Dominate the Logic.

"Abstract theory is just a movement we haven't seen yet."

Subscribe for Full Proofs Early Access

Visualizing...

The Formal Theorem

Analytical Intuition.

Institutional Warning.

Academic Inquiries.

Why is the threshold for Cook's distance often set at 4/n?

Can influence diagnostics be used in models with non-constant variance?

What happens to DFBETAS when the design matrix is collinear?

Standardized References.

Related Proofs Cluster.

The Matrix Formulation of the General Linear Model: Y = Xβ + ϵ and its Fundamental Assumptions

Derivation of the Ordinary Least Squares (OLS) Estimator: β̂ = (X'X)⁻¹X'Y

Proof of Unbiasedness of the OLS Estimator: E(β̂) = β

Derivation of the Variance-Covariance Matrix of the OLS Estimator: Var(β̂) = σ²(X'X)⁻¹

Institutional Citation

Dominate the Logic.