Meta-regression

Meta-regression is a tool used in meta-analysis to examine the impact of moderator variables on study effect size using regression-based techniques. Meta-regression is more effective at this task than are standard meta-analytic techniques.[1]

Meta-regression models

Meta-regression analysis (MRA) is a quantitative method of conducting literature surveys. Meta-regression has gained popularity in social, behavioral and economic sciences. Important applications have focused on qualifying estimates of policy-relevant parameters, testing economic theories, explaining heterogeneity, and qualifying potential biases. Generally, three types of models can be distinguished in the literature on meta-analysis: simple regression, fixed effect meta-regression and random effects meta-regression.

Simple regression

The model can be specified as

${\displaystyle y_{j}=\beta _{0}+\beta _{1}x_{1j}+\beta _{2}x_{2j}+\cdots +\varepsilon }$

Where ${\displaystyle y_{j}}$ is the effect size in study ${\displaystyle j}$ and ${\displaystyle \beta _{0}}$ (intercept) the estimated overall effect size. The variables ${\displaystyle x_{i.}(i=1\ldots k)}$ specify different characteristics of the study, ${\displaystyle \varepsilon }$ specifies the between study variation. Note that this model does not allow specification of within study variation.

Fixed-effect meta-regression

Fixed-effect meta-regression assumes that the sampled effect size ${\displaystyle \theta }$ is normally distributed with ${\displaystyle {\mathcal {N}}(\theta ,\sigma _{\theta })}$ where ${\displaystyle \sigma _{\theta }^{2}}$ is the within-study variance of the effect size. A fixed-effect meta-regression model thus allows for within-study variability but not between-study variability because all studies have an identical expected fixed effect size ${\displaystyle \theta }$, i.e. ${\displaystyle \varepsilon =0}$.

${\displaystyle y_{j}=\beta _{0}+\beta _{1}x_{1j}+\beta _{2}x_{2j}+\cdots +\eta _{j}\,}$

Here ${\displaystyle \sigma _{\eta _{j}}^{2}}$ is the variance of the effect size in study ${\displaystyle j}$. Fixed effect meta-regression ignores between study variation. As a result, parameter estimates are biased if between study variation can not be ignored. Furthermore, generalizations to the population are not possible.

Random effects meta-regression

Random effects meta-regression rests on the assumption that ${\displaystyle \theta }$ in ${\displaystyle {\mathcal {N}}(\theta ,\sigma _{i})}$ is a random variable following a (hyper-)distribution ${\displaystyle {\mathcal {N}}(\theta ,\sigma _{\theta }).}$ A random effects meta-regression is called a mixed effects model when moderators are added to the model.

${\displaystyle y_{j}=\beta _{0}+\beta _{1}x_{1j}+\beta _{2}x_{2j}+\cdots +\eta +\varepsilon _{j}\,}$

Here ${\displaystyle \sigma _{\varepsilon _{j}}^{2}}$ is the variance of the effect size in study ${\displaystyle j}$. Between study variance ${\displaystyle \sigma _{\eta }^{2}}$ is estimated using common estimation procedures for random effects models (restricted maximum likelihood (REML) estimators).

Which model to choose

Meta-regression has been employed as a technique to derive improved parameter estimates that are of direct use to policy makers. Meta-regression provides a framework for replication and offers a sensitivity analysis for model specification.[2] There are a number of strategies for identifying and coding empirical observational data. Meta-regression models can be extended for modeling within-study dependence, excess heterogeneity and publication selection.[2] The simple regression model does not allow for within study variation. The fixed effects regression model does not allow for between study variation. The random or mixed effects model allows for within study variation and between study variation and is therefore the most appropriate model to choose in many applications. Whether there is between study variation (excess heterogeneity) can be tested under the assumption that effect sizes are homogeneous or have a tendency to a central mean. If the test shows that the effect sizes have excess heterogeneity, the random effects meta-regression model may be most appropriate.

Applications

Meta-regression is an objective and statistically rigorous approach to systematic reviews. Recent applications include quantitative reviews of the empirical literature in economics, business, energy and water policy.[3] Meta-regression analyses have been seen in studies of price and income elasticities of various commodities and taxes,[3] productivity spillovers on multinational companies,[4] and calculations on the value of a statistical life (VSL).[5] Other recent meta-regression analyses have focused on qualifying elasticities derived from demand functions. Examples include own price elasticities for alcohol, tobacco, water and energy.[3]

In energy conservation, meta-regression analysis has been used to evaluate behavioral information strategies in the residential electricity sector.[6] In water policy analysis, meta-regression has been used to evaluate cost savings estimates due to privatization of local government services for water distribution and solid waste collection.[7] Meta-regression is an increasingly popular tool to evaluate the available evidence in cost-benefit analysis studies of a policy or program spread across multiple studies.

References

1. Huizenga, Hilde M.; Visser, Ingmar; Dolan, Conor V. (February 2011). "Testing overall and moderator effects in random effects meta-regression: Random effects meta-regression". British Journal of Mathematical and Statistical Psychology. 64 (1): 1–19. doi:10.1348/000711010X522687. PMID 21506942. Random effects meta‐analysis in its unconditional form (i.e., without moderators) does accommodate within‐ and between‐study variation, but it is limited to tests of overall effects. There are, however, several ad hoc procedures to test moderator effects in follow‐up analyses...Random effects meta‐regression is generally preferable to these ad hoc procedures, because it does not suffer from these drawbacks.
2. T.D. Stanley and Stephen B. Jarrell, (1989). Meta-regression analysis: A quantitative method of literature surveys. Journal of Economic Surveys, 19(3) 299-308.
3. T.D. Stanley and Hristos Doucouliagos (2009). Meta-regression Analysis in Economics and Business, New York: Routledge.
4. H. Gorg and Eric Strobl (2001). Multinational companies and productivity spillovers: A meta-analysis. The Economic Journal, 111(475) 723-739.
5. F. Bellavance, Georges Dionne, and Martin Lebeau (2009). The value of a statistical life: A meta-analysis with a mixed effects regression model, Journal of Health Economics, 28(2) 444-464.
6. M.A. Delmas, Miriam Fischlein and Omar I. Asensio (2013). Information strategies and energy conservation behavior: A meta-analysis of experimental studies 1975-2012. Energy Policy, 61, 729-739.
7. G. Bel, Xavier Fageda and Mildred E. Warner (2010). Is private production of public services cheaper than public production? A meta-regression analysis of solid waste and water services. Journal of Policy Analysis and Management. 29(3), 553-577.

Further reading

• Thompson, S. G.; Higgins, J. P. T. (2002). "How should meta-regression analyses be undertaken and interpreted?". Statistics in Medicine. 21 (11): 1559–1573. doi:10.1002/sim.1187. PMID 12111920.
• Roberts, Colin; Stanley, T. D. (2005). Meta-Regression Analysis: Issues of Publication Bias in Economics. Wiley-Blackwell. ISBN 978-1-4051-3799-7.
• Bonett DG (2009). "Meta-analytic interval estimation for standardized and unstandardized mean differences". Psychological Methods. 14 (3): 225–38. doi:10.1037/a0016619. PMID 19719359.