IDEAS home Printed from https://ideas.repec.org/p/nbr/nberwo/20673.html
   My bibliography  Save this paper

Measuring the Sensitivity of Parameter Estimates to Estimation Moments

Author

Listed:
  • Isaiah Andrews
  • Matthew Gentzkow
  • Jesse M. Shapiro

Abstract

We propose a local measure of the relationship between parameter estimates and the moments of the data they depend on. Our measure can be computed at negligible cost even for complex structural models. We argue that reporting this measure can increase the transparency of structural estimates, making it easier for readers to predict the way violations of identifying assumptions would affect the results. When the key assumptions are orthogonality between error terms and excluded instruments, we show that our measure provides a natural extension of the omitted variables bias formula for nonlinear models. We illustrate with applications to published articles in several fields of economics.

Suggested Citation

  • Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2014. "Measuring the Sensitivity of Parameter Estimates to Estimation Moments," NBER Working Papers 20673, National Bureau of Economic Research, Inc.
  • Handle: RePEc:nbr:nberwo:20673
    Note: CF IO ME PE TWP
    as

    Download full text from publisher

    File URL: http://www.nber.org/papers/w20673.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    2. Hansen, Lars Peter, 1982. "Large Sample Properties of Generalized Method of Moments Estimators," Econometrica, Econometric Society, vol. 50(4), pages 1029-1054, July.
    3. Stefano DellaVigna & John A. List & Ulrike Malmendier, 2012. "Testing for Altruism and Social Pressure in Charitable Giving," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 127(1), pages 1-56.
    4. Liran Einav & Amy Finkelstein & Stephen P. Ryan & Paul Schrimpf & Mark R. Cullen, 2013. "Selection on Moral Hazard in Health Insurance," American Economic Review, American Economic Association, vol. 103(1), pages 178-219, February.
    5. Ronchetti, Elvezio & Trojani, Fabio, 2001. "Robust inference with GMM estimators," Journal of Econometrics, Elsevier, vol. 101(1), pages 37-69, March.
    6. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    7. Boris Nikolov & Toni M. Whited, 2014. "Agency Conflicts and Cash: Estimates from a Dynamic Model," Journal of Finance, American Finance Association, vol. 69(5), pages 1883-1921, October.
    8. Raj Chetty, 2009. "Sufficient Statistics for Welfare Analysis: A Bridge Between Structural and Reduced-Form Methods," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 451-488, May.
    9. Mariacristina De Nardi & Eric French & John B. Jones, 2010. "Why Do the Elderly Save? The Role of Medical Expenses," Journal of Political Economy, University of Chicago Press, vol. 118(1), pages 39-75, February.
    10. Steven D. Levitt & John A. List & Chad Syverson, 2013. "Toward an Understanding of Learning by Doing: Evidence from an Automobile Assembly Plant," Journal of Political Economy, University of Chicago Press, vol. 121(4), pages 643-681.
    11. Matthew Gentzkow, 2007. "Valuing New Goods in a Model with Complementarity: Online Newspapers," American Economic Review, American Economic Association, vol. 97(3), pages 713-744, June.
    12. Levon Barseghyan & Francesca Molinari & Ted O'Donoghue & Joshua C. Teitelbaum, 2013. "The Nature of Risk Preferences: Evidence from Insurance Choices," American Economic Review, American Economic Association, vol. 103(6), pages 2499-2529, October.
    13. Berkowitz, Daniel & Caner, Mehmet & Fang, Ying, 2008. "Are "Nearly Exogenous Instruments" reliable?," Economics Letters, Elsevier, vol. 101(1), pages 20-23, October.
    14. Gourieroux, C & Monfort, A & Renault, E, 1993. "Indirect Inference," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 8(S), pages 85-118, Suppl. De.
    15. Steven Berry & Alon Eizenberg & Joel Waldfogel, 2016. "Optimal product variety in radio markets," RAND Journal of Economics, RAND Corporation, vol. 47(3), pages 463-497, August.
    16. Lawrence J. Christiano & Martin Eichenbaum & Charles L. Evans, 2005. "Nominal Rigidities and the Dynamic Effects of a Shock to Monetary Policy," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 1-45, February.
    17. Kei Kawai & Yasutora Watanabe, 2013. "Inferring Strategic Voting," American Economic Review, American Economic Association, vol. 103(2), pages 624-662, April.
    18. Kristensen, Dennis & Salanié, Bernard, 2017. "Higher-order properties of approximate estimators," Journal of Econometrics, Elsevier, vol. 198(2), pages 189-208.
    19. M. Kate Bundorf & Jonathan Levin & Neale Mahoney, 2012. "Pricing and Welfare in Health Plan Choice," American Economic Review, American Economic Association, vol. 102(7), pages 3214-3248, December.
    20. Sonia Jaffe & E. Glen Weyl, 2013. "The First-Order Approach to Merger Analysis," American Economic Journal: Microeconomics, American Economic Association, vol. 5(4), pages 188-218, November.
    21. repec:feb:framed:0087 is not listed on IDEAS
    22. Matthew Gentzkow & Jesse M. Shapiro & Michael Sinkinson, 2014. "Competition and Ideological Diversity: Historical Evidence from US Newspapers," American Economic Review, American Economic Association, vol. 104(10), pages 3073-3114, October.
    23. Jae-On Kim & G. Donald Ferree Jr., 1981. "Standardization in Causal Analysis," Sociological Methods & Research, , vol. 10(2), pages 187-210, November.
    24. James J. Heckman, 2010. "Building Bridges between Structural and Program Evaluation Approaches to Evaluating Policy," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 356-398, June.
    25. Ying Fan, 2013. "Ownership Consolidation and Product Characteristics: A Study of the US Daily Newspaper Market," American Economic Review, American Economic Association, vol. 103(5), pages 1598-1628, August.
    26. Aviv Nevo, 2000. "Mergers with Differentiated Products: The Case of the Ready-to-Eat Cereal Industry," RAND Journal of Economics, The RAND Corporation, vol. 31(3), pages 395-421, Autumn.
    27. Pierre-Olivier Gourinchas & Jonathan A. Parker, 2002. "Consumption Over the Life Cycle," Econometrica, Econometric Society, vol. 70(1), pages 47-89, January.
    28. Gregory S. Crawford & Ali Yurukoglu, 2012. "The Welfare Effects of Bundling in Multichannel Television Markets," American Economic Review, American Economic Association, vol. 102(2), pages 643-685, April.
    29. Timothy G. Conley & Christian B. Hansen & Peter E. Rossi, 2012. "Plausibly Exogenous," The Review of Economics and Statistics, MIT Press, vol. 94(1), pages 260-272, February.
    30. Ronald L. Goettler & Brett R. Gordon, 2011. "Does AMD Spur Intel to Innovate More?," Journal of Political Economy, University of Chicago Press, vol. 119(6), pages 1141-1200.
    31. Guggenberger, Patrik, 2012. "On The Asymptotic Size Distortion Of Tests When Instruments Locally Violate The Exogeneity Assumption," Econometric Theory, Cambridge University Press, vol. 28(2), pages 387-421, April.
    32. Liran Einav & Amy Finkelstein & Mark R. Cullen, 2010. "Estimating Welfare in Insurance Markets Using Variation in Prices," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(3), pages 877-921.
    33. Leamer, Edward E, 1983. "Let's Take the Con Out of Econometrics," American Economic Review, American Economic Association, vol. 73(1), pages 31-43, March.
    34. Alessandra Voena, 2015. "Yours, Mine, and Ours: Do Divorce Laws Affect the Intertemporal Behavior of Married Couples?," American Economic Review, American Economic Association, vol. 105(8), pages 2295-2332, August.
    35. Amil Petrin, 2002. "Quantifying the Benefits of New Products: The Case of the Minivan," Journal of Political Economy, University of Chicago Press, vol. 110(4), pages 705-729, August.
    36. Rosa L. Matzkin, 2013. "Nonparametric Identification in Structural Economic Models," Annual Review of Economics, Annual Reviews, vol. 5(1), pages 457-486, May.
    37. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    38. Greg Kaplan, 2012. "Moving Back Home: Insurance against Labor Market Risk," Journal of Political Economy, University of Chicago Press, vol. 120(3), pages 446-512.
    39. Liran Einav & Amy Finkelstein & Paul Schrimpf, 2015. "The Response of Drug Expenditure to Nonlinear Contract Design: Evidence from Medicare Part D," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(2), pages 841-899.
    40. Melanie Morten, 2016. "Temporary Migration and Endogenous Risk Sharing in Village India," NBER Working Papers 22159, National Bureau of Economic Research, Inc.
    41. Jean‐Pierre Dubé & Jeremy T. Fox & Che‐Lin Su, 2012. "Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation," Econometrica, Econometric Society, vol. 80(5), pages 2231-2267, September.
    42. Aviv Nevo & Adam M. Rosen, 2012. "Identification With Imperfect Instruments," The Review of Economics and Statistics, MIT Press, vol. 94(3), pages 659-671, August.
    43. Ariel Pakes, 2003. "Common Sense and Simplicity in Empirical Industrial Organization," Review of Industrial Organization, Springer;The Industrial Organization Society, vol. 23(3), pages 193-215, December.
    44. Christopher R. Knittel & Konstantinos Metaxoglou, 2014. "Estimation of Random-Coefficient Demand Models: Two Empiricists' Perspective," The Review of Economics and Statistics, MIT Press, vol. 96(1), pages 34-59, March.
    45. Smith, A A, Jr, 1993. "Estimating Nonlinear Time-Series Models Using Simulated Vector Autoregressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 8(S), pages 63-84, Suppl. De.
    46. Robin S. Lee, 2013. "Vertical Integration and Exclusivity in Platform and Two-Sided Markets," American Economic Review, American Economic Association, vol. 103(7), pages 2960-3000, December.
    47. Michaela Draganska & Michael Mazzeo & Katja Seim, 2009. "Beyond plain vanilla: Modeling joint product assortment and pricing decisions," Quantitative Marketing and Economics (QME), Springer, vol. 7(2), pages 105-146, June.
    48. K. Newey, Whitney, 1985. "Generalized method of moments specification testing," Journal of Econometrics, Elsevier, vol. 29(3), pages 229-256, September.
    49. Xiaohong Chen & Elie Tamer & Alexander Torgovitsky, 2011. "Sensitivity Analysis in Semiparametric Likelihood Models," Cowles Foundation Discussion Papers 1836, Cowles Foundation for Research in Economics, Yale University.
    50. Yuichi Kitamura & Taisuke Otsu & Kirill Evdokimov, 2013. "Robustness, Infinitesimal Neighborhoods, and Moment Restrictions," Econometrica, Econometric Society, vol. 81(3), pages 1185-1201, May.
    51. Michael J. Mazzeo, 2002. "Product Choice and Oligopoly Market Structure," RAND Journal of Economics, The RAND Corporation, vol. 33(2), pages 221-242, Summer.
    52. David Berger & Joseph Vavra, 2015. "Consumption Dynamics During Recessions," Econometrica, Econometric Society, vol. 83, pages 101-154, January.
    53. Joseph G. Altonji & Todd E. Elder & Christopher R. Taber, 2005. "An Evaluation of Instrumental Variable Strategies for Estimating the Effects of Catholic Schooling," Journal of Human Resources, University of Wisconsin Press, vol. 40(4), pages 791-821.
    54. Claire S. H. Lim, 2013. "Preferences and Incentives of Appointed and Elected Public Officials: Evidence from State Trial Court Judges," American Economic Review, American Economic Association, vol. 103(4), pages 1360-1397, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "Transparency in Structural Research," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 38(4), pages 711-722, October.
    2. Isaiah Andrews & Matthew Gentzkow & Jesse M. Shapiro, 2020. "On the Informativeness of Descriptive Statistics for Structural Estimates," Econometrica, Econometric Society, vol. 88(6), pages 2231-2258, November.
    3. David P. Byrne & Susumu Imai & Vasilis Sarafidis, 2015. "Instrument-free Identifcation and Estimation of the Diferentiated Products Models," Department of Economics - Working Papers Series 1198, The University of Melbourne.
    4. Victor Aguirregabiria & Margaret Slade, 2017. "Empirical models of firms and industries," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 50(5), pages 1445-1488, December.
    5. Timothy B. Armstrong & Michal Kolesár, 2021. "Sensitivity analysis using approximate moment condition models," Quantitative Economics, Econometric Society, vol. 12(1), pages 77-108, January.
    6. Friberg, Richard & Romahn, André, 2015. "Divestiture requirements as a tool for competition policy: A case from the Swedish beer market," International Journal of Industrial Organization, Elsevier, vol. 42(C), pages 1-18.
    7. Christopher Conlon & Julie Holland Mortimer, 2021. "Empirical properties of diversion ratios," RAND Journal of Economics, RAND Corporation, vol. 52(4), pages 693-726, December.
    8. Martin Gaynor & Kate Ho & Robert J. Town, 2015. "The Industrial Organization of Health-Care Markets," Journal of Economic Literature, American Economic Association, vol. 53(2), pages 235-284, June.
    9. Matthew Gentzkow & Jesse M. Shapiro & Michael Sinkinson, 2014. "Competition and Ideological Diversity: Historical Evidence from US Newspapers," American Economic Review, American Economic Association, vol. 104(10), pages 3073-3114, October.
    10. Byrne, David P. & Imai, Susumu & Jain, Neelam & Sarafidis, Vasilis, 2022. "Instrument-free identification and estimation of differentiated products models using cost data," Journal of Econometrics, Elsevier, vol. 228(2), pages 278-301.
    11. Gautam Gowrisankaran & Marc Rysman, 2012. "Dynamics of Consumer Demand for New Durable Goods," Journal of Political Economy, University of Chicago Press, vol. 120(6), pages 1173-1219.
    12. Byrne, D. P. & Imai, S. & Jain, N. & Sarafidis, V. & Hirukawa, M., 2020. "Identification and Estimation of Differentiated Products Models using Cost Data," Working Papers 15/05, Department of Economics, City University London.
    13. Wang, Ao, 2021. "A BLP Demand Model of Product-Level Market Shares with Complementarity," The Warwick Economics Research Paper Series (TWERPS) 1351, University of Warwick, Department of Economics.
    14. Bokhari, Farasat A.S. & Mariuzzo, Franco, 2018. "Demand estimation and merger simulations for drugs: Logits v. AIDS," International Journal of Industrial Organization, Elsevier, vol. 61(C), pages 653-685.
    15. Miravete, Eugenio J. & Seim, Katja & Thurk, Jeff, 2023. "Pass-through and tax incidence in differentiated product markets," International Journal of Industrial Organization, Elsevier, vol. 90(C).
    16. Iaria, Alessandro & ,, 2020. "Identification and Estimation of Demand for Bundles," CEPR Discussion Papers 14363, C.E.P.R. Discussion Papers.
    17. Lopez, Rigoberto A. & Mohapatra, Debashrita & Steinbach, Sandro, 2024. "Mergers, store locations, and jobs: Evidence from the food retail industry," 2024 Annual Meeting, July 28-30, New Orleans, LA 343711, Agricultural and Applied Economics Association.
    18. Gaynor, Martin & Town, Robert J., 2011. "Competition in Health Care Markets," Handbook of Health Economics, in: Mark V. Pauly & Thomas G. Mcguire & Pedro P. Barros (ed.), Handbook of Health Economics, volume 2, chapter 0, pages 499-637, Elsevier.
    19. Daniel Chaves, 2022. "Market Power, Taxation and Product Variety in the Brazilian Automobile Industry," University of Western Ontario, Departmental Research Report Series 20227, University of Western Ontario, Department of Economics.
    20. Christopher Conlon & Jeff Gortmaker, 2020. "Best practices for differentiated products demand estimation with PyBLP," RAND Journal of Economics, RAND Corporation, vol. 51(4), pages 1108-1161, December.

    More about this item

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • C52 - Mathematical and Quantitative Methods - - Econometric Modeling - - - Model Evaluation, Validation, and Selection

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:nbr:nberwo:20673. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://edirc.repec.org/data/nberrus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.