Heteroscedasticity-consistent standard errors

The topic of heteroscedasticity-consistent (HC) standard errors arises in statistics and econometrics in the context of linear regression and time series analysis. These are also known as Eicker–Huber–White standard errors (also Huber–White standard errors or White standard errors),[1] to recognize the contributions of Friedhelm Eicker,[2] Peter J. Huber,[3] and Halbert White.[4]

In regression and time-series modelling, basic forms of models make use of the assumption that the errors or disturbances u_i have the same variance across all observation points. When this is not the case, the errors are said to be heteroscedastic, or to have heteroscedasticity, and this behaviour will be reflected in the residuals ${\displaystyle {\widehat {u}}_{i}}$ estimated from a fitted model. Heteroscedasticity-consistent standard errors are used to allow the fitting of a model that does contain heteroscedastic residuals. The first such approach was proposed by Huber (1967), and further improved procedures have been produced since for cross-sectional data, time-series data and GARCH estimation.

Heteroscedasticity-consistent standard errors that differ from classical standard errors is an indicator of model misspecification. This misspecification is not fixed by merely replacing the classical with heteroscedasticity-consistent standard errors; for all but a few quantities of interest, the misspecification may lead to bias. In most situations, the problem should be found and fixed.[5] Other types of standard error adjustments, such as clustered standard errors, may be considered as extensions to HC standard errors.

History

Heteroscedasticity-consistent standard errors are introduced by Friedhelm Eicker[6],[7] and popularized in econometrics by Halbert White.

Problem

Assume that we are studying the linear regression model

${\displaystyle Y=X\beta +U,\,}$

where X is the vector of explanatory variables and β is a k × 1 column vector of parameters to be estimated.

The ordinary least squares (OLS) estimator is

${\displaystyle {\widehat {\beta }}_{\text{OLS}}=(\mathbb {X} '\mathbb {X} )^{-1}\mathbb {X} '\mathbb {Y} .\,}$

where ${\displaystyle \mathbb {X} }$ denotes the matrix of stacked ${\displaystyle X_{i}'}$ values observed in the data.

If the sample errors have equal variance σ² and are uncorrelated, then the least-squares estimate of β is BLUE (best linear unbiased estimator), and its variance is easily estimated with

${\displaystyle v_{\text{OLS}}\left[{\widehat {\beta }}_{\text{OLS}}\right]=s^{2}(\mathbb {X} '\mathbb {X} )^{-1},\quad s^{2}={\frac {\sum _{i}{\widehat {u}}_{i}^{2}}{n-k}}}$

where ${\displaystyle {\widehat {u}}_{i}=Y_{i}-X_{i}{\widehat {\beta }}_{\text{OLS}}}$ are the regression residuals.

When the assumptions of ${\displaystyle \operatorname {E} [uu']=\sigma ^{2}I_{n}}$ are violated, the OLS estimator loses its desirable properties. Indeed,

${\displaystyle V\left[{\widehat {\beta }}_{\text{OLS}}\right]=V[(\mathbb {X} '\mathbb {X} )^{-1}\mathbb {X} '\mathbb {Y} ]=(\mathbb {X} '\mathbb {X} )^{-1}\mathbb {X} '\Sigma \mathbb {X} (\mathbb {X} '\mathbb {X} )^{-1}}$

where ${\displaystyle \Sigma =V[u].}$

While the OLS point estimator remains unbiased, it is not "best" in the sense of having minimum mean square error, and the OLS variance estimator ${\displaystyle v_{\text{OLS}}\left[{\widehat {\beta }}_{\text{OLS}}\right]}$ does not provide a consistent estimate of the variance of the OLS estimates.

For any non-linear model (for instance logit and probit models), however, heteroscedasticity has more severe consequences: the maximum likelihood estimates of the parameters will be biased (in an unknown direction), as well as inconsistent (unless the likelihood function is modified to correctly take into account the precise form of heteroscedasticity).[8][9] As pointed out by Greene, “simply computing a robust covariance matrix for an otherwise inconsistent estimator does not give it redemption.”[10]

Solution

If the regression errors ${\displaystyle u_{i}}$ are independent, but have distinct variances σ_i², then ${\displaystyle \Sigma =\operatorname {diag} (\sigma _{1}^{2},\ldots ,\sigma _{n}^{2})}$ which can be estimated with ${\displaystyle {\widehat {\sigma }}_{i}^{2}={\widehat {u}}_{i}^{2}}$. This provides White's (1980) estimator, often referred to as HCE (heteroscedasticity-consistent estimator):

${\displaystyle {\begin{aligned}v_{\text{HCE}}\left[{\widehat {\beta }}_{\text{OLS}}\right]&={\frac {1}{n}}\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'\right)^{-1}\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'{\widehat {u}}_{i}^{2}\right)\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'\right)^{-1}\\&=(\mathbb {X} '\mathbb {X} )^{-1}(\mathbb {X} '\operatorname {diag} ({\widehat {u}}_{1}^{2},\ldots ,{\widehat {u}}_{n}^{2})\mathbb {X} )(\mathbb {X} '\mathbb {X} )^{-1},\end{aligned}}}$

where as above ${\displaystyle \mathbb {X} }$ denotes the matrix of stacked ${\displaystyle X_{i}'}$ values from the data. The estimator can be derived in terms of the generalized method of moments (GMM).

Note that also often discussed in the literature (including in White's paper itself) is the covariance matrix ${\displaystyle {\widehat {\Omega }}_{n}}$ of the ${\displaystyle {\sqrt {n}}}$-consistent limiting distribution:

${\displaystyle {\sqrt {n}}({\widehat {\beta }}_{n}-\beta )\,{\xrightarrow {d}}\,N(0,\Omega ),}$

where

${\displaystyle \Omega =\operatorname {E} [XX']^{-1}\operatorname {Var} [Xu]\operatorname {E} [XX']^{-1},}$

and

${\displaystyle {\begin{aligned}{\widehat {\Omega }}_{n}&=\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'\right)^{-1}\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'{\widehat {u}}_{i}^{2}\right)\left({\frac {1}{n}}\sum _{i}X_{i}X_{i}'\right)^{-1}\\&=n(\mathbb {X} '\mathbb {X} )^{-1}(\mathbb {X} '\operatorname {diag} ({\widehat {u}}_{1}^{2},\ldots ,{\widehat {u}}_{n}^{2})\mathbb {X} )(\mathbb {X} '\mathbb {X} )^{-1}\end{aligned}}}$

Thus,

${\displaystyle {\widehat {\Omega }}_{n}=n\cdot v_{\text{HCE}}[{\widehat {\beta }}_{\text{OLS}}]}$

and

${\displaystyle {\widehat {\operatorname {Var} }}[Xu]={\frac {1}{n}}\sum _{i}X_{i}X_{i}'{\widehat {u}}_{i}^{2}={\frac {1}{n}}\mathbb {X} '\operatorname {diag} ({\widehat {u}}_{1}^{2},\ldots ,{\widehat {u}}_{n}^{2})\mathbb {X} .}$

Precisely which covariance matrix is of concern is a matter of context.

Alternative estimators have been proposed in MacKinnon & White (1985) that correct for unequal variances of regression residuals due to different leverage.[11] Unlike the asymptotic White's estimator, their estimators are unbiased when the data are homoscedastic.

See also

• Delta method
• Generalized least squares
• Generalized estimating equations
• Weighted least squares, an alternative formulation
• White test — a test for whether heteroscedasticity is present.
• Newey–West estimator
• Quasi-maximum likelihood estimate

Software

• EViews: EViews version 8 offers three different methods for robust least squares: M-estimation (Huber, 1973), S-estimation (Rousseeuw and Yohai, 1984), and MM-estimation (Yohai 1987).[12]
• MATLAB: See the hac function in the Econometrics toolbox.[13]
• Python: The Statsmodel package offers various robust standard error estimates, see statsmodels.regression.linear_model.RegressionResults for further descriptions
• R: the vcovHC() command from the sandwich package.[14][15]
• RATS: robusterrors option is available in many of the regression and optimization commands (linreg, nlls, etc.).
• Stata: robust option applicable in many pseudo-likelihood based procedures.[16]
• Gretl: the option --robust to several estimation commands (such as ols) in the context of a cross-sectional dataset produces robust standard errors.[17]

References

1. Kleiber, C.; Zeileis, A. (2006). "Applied Econometrics with R" (PDF). UseR-2006 conference. Archived from the original (PDF) on April 22, 2007.
2. Eicker, Friedhelm (1967). "Limit Theorems for Regression with Unequal and Dependent Errors". Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. pp. 59–82. MR 0214223. Zbl 0217.51201.
3. Huber, Peter J. (1967). "The behavior of maximum likelihood estimates under nonstandard conditions". Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. pp. 221–233. MR 0216620. Zbl 0212.21504.
4. White, Halbert (1980). "A Heteroscedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroscedasticity". Econometrica. 48 (4): 817–838. CiteSeerX 10.1.1.11.7646. doi:10.2307/1912934. JSTOR 1912934. MR 0575027.
5. King, Gary; Roberts, Margaret E. (2015). "How Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It". Political Analysis. 23 (2): 159–179. doi:10.1093/pan/mpu015. ISSN 1047-1987.
6. "Asymptotic Normality and Consistency of the Least Squares Estimators for Families of Linear Regressions". Cite journal requires |journal= (help)
7. "Limit theorems for regressions with unequal and dependent errors". Cite journal requires |journal= (help)
8. Giles, Dave (May 8, 2013). "Robust Standard Errors for Nonlinear Models". Econometrics Beat.
9. Guggisberg, Michael (2019). "Misspecified Discrete Choice Models and Huber-White Standard Errors". Journal of Econometric Methods. 8 (1). doi:10.1515/jem-2016-0002.
10. Greene, William H. (2012). Econometric Analysis (Seventh ed.). Boston: Pearson Education. pp. 692–693. ISBN 978-0-273-75356-8.
11. MacKinnon, James G.; White, Halbert (1985). "Some Heteroskedastic-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties". Journal of Econometrics. 29 (3): 305–325. doi:10.1016/0304-4076(85)90158-7. hdl:10419/189084.
12. http://www.eviews.com/EViews8/ev8ecrobust_n.html
13. "Heteroscedasticity and autocorrelation consistent covariance estimators". Econometrics Toolbox.
14. sandwich: Robust Covariance Matrix Estimators
15. Kleiber, Christian; Zeileis, Achim (2008). Applied Econometrics with R. New York: Springer. pp. 106–110. ISBN 978-0-387-77316-2.
16. See online help for _robust option and regress command.
17. "Robust covariance matrix estimation" (PDF). Gretl User's Guide, chapter 19.

Further reading

• Freedman, David A. (2006). "On The So-Called 'Huber Sandwich Estimator' and 'Robust Standard Errors'". The American Statistician. 60 (4): 299–302. doi:10.1198/000313006X152207.
• Hardin, James W. (2003). "The Sandwich Estimate of Variance". In Fomby, Thomas B.; Hill, R. Carter (eds.). Maximum Likelihood Estimation of Misspecified Models: Twenty Years Later. Amsterdam: Elsevier. pp. 45–74. ISBN 0-7623-1075-8.
• Hayes, Andrew F.; Cai, Li (2007). "Using heteroscedasticity-consistent standard error estimators in OLS regression: An introduction and software implementation". Behavior Research Methods. 39 (4): 709–722. doi:10.3758/BF03192961. PMID 18183883.
• King, Gary; Roberts, Margaret E. (2015). "How Robust Standard Errors Expose Methodological Problems They Do Not Fix, and What to Do About It". Political Analysis. 23 (2): 159–179. doi:10.1093/pan/mpu015.
• Wooldridge, Jeffrey M. (2009). "Heteroskedasticity-Robust Inference after OLS Estimation". Introductory Econometrics : A Modern Approach (Fourth ed.). Mason: South-Western. pp. 265–271. ISBN 978-0-324-66054-8.