IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1712.07248.html
   My bibliography  Save this paper

Towards a General Large Sample Theory for Regularized Estimators

Author

Listed:
  • Michael Jansson
  • Demian Pouzo

Abstract

We present a general framework for studying regularized estimators; such estimators are pervasive in estimation problems wherein "plug-in" type estimators are either ill-defined or ill-behaved. Within this framework, we derive, under primitive conditions, consistency and a generalization of the asymptotic linearity property. We also provide data-driven methods for choosing tuning parameters that, under some conditions, achieve the aforementioned properties. We illustrate the scope of our approach by presenting a wide range of applications.

Suggested Citation

  • Michael Jansson & Demian Pouzo, 2017. "Towards a General Large Sample Theory for Regularized Estimators," Papers 1712.07248, arXiv.org, revised Jul 2020.
    • Handle: RePEc:arx:papers:1712.07248
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1712.07248
    File Function: Latest version
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Daniel Ackerberg & Xiaohong Chen & Jinyong Hahn & Zhipeng Liao, 2014. "Asymptotic Efficiency of Semiparametric Two-step GMM," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 81(3), pages 919-943.
    2. Peter Bickel & Bo Li & Alexandre Tsybakov & Sara Geer & Bin Yu & Teófilo Valdés & Carlos Rivero & Jianqing Fan & Aad Vaart, 2006. "Regularization in statistics," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 15(2), pages 271-344, September.
    3. Xiaohong Chen & Demian Pouzo, 2015. "Sieve Wald and QLR Inferences on Semi/Nonparametric Conditional Moment Models," Econometrica, Econometric Society, vol. 83(3), pages 1013-1079, May.
    4. Powell, James L. & Stoker, Thomas M., 1996. "Optimal bandwidth choice for density-weighted averages," Journal of Econometrics, Elsevier, vol. 75(2), pages 291-316, December.
    5. Jin Seo Cho & Halbert White, 2017. "Supplements to "Directionally Differentiable Econometric Models"," Working papers 2017rwp-103a, Yonsei University, Yonsei Economics Research Institute.
    6. Donald W. K. Andrews, 2000. "Inconsistency of the Bootstrap when a Parameter Is on the Boundary of the Parameter Space," Econometrica, Econometric Society, vol. 68(2), pages 399-406, March.
    7. Belloni, Alexandre & Chernozhukov, Victor & Chetverikov, Denis & Kato, Kengo, 2015. "Some new asymptotic theory for least squares series: Pointwise and uniform results," Journal of Econometrics, Elsevier, vol. 186(2), pages 345-366.
    8. Severini, Thomas A. & Tripathi, Gautam, 2012. "Efficiency bounds for estimating linear functionals of nonparametric regression models with endogenous regressors," Journal of Econometrics, Elsevier, vol. 170(2), pages 491-498.
    9. Xiaohong Chen & Timothy M. Christensen, 2018. "Optimal sup‐norm rates and uniform inference on nonlinear functionals of nonparametric IV regression," Quantitative Economics, Econometric Society, vol. 9(1), pages 39-84, March.
    10. S. Darolles & Y. Fan & J. P. Florens & E. Renault, 2011. "Nonparametric Instrumental Regression," Econometrica, Econometric Society, vol. 79(5), pages 1541-1565, September.
    11. Whitney K. Newey & Fushing Hsieh & James M. Robins, 2004. "Twicing Kernels and a Small Bias Property of Semiparametric Estimators," Econometrica, Econometric Society, vol. 72(3), pages 947-962, May.
    12. Hall, Alastair R. & Inoue, Atsushi, 2003. "The large sample behaviour of the generalized method of moments estimator in misspecified models," Journal of Econometrics, Elsevier, vol. 114(2), pages 361-394, June.
    13. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    14. Xiaohong Chen & Timothy Christensen, 2013. "Optimal Sup-norm Rates, Adaptivity and Inference in Nonparametric Instrumental Variables Estimation," Cowles Foundation Discussion Papers 1923R, Cowles Foundation for Research in Economics, Yale University, revised Apr 2015.
    15. Gautier, Eric & Gaillac, Christophe, 2019. "Adaptive estimation in the linear random coefficients model when regressors have limited variation," TSE Working Papers 19-1026, Toulouse School of Economics (TSE).
    16. Cattaneo, Matias D. & Farrell, Max H., 2013. "Optimal convergence rates, Bahadur representation, and asymptotic normality of partitioning estimators," Journal of Econometrics, Elsevier, vol. 174(2), pages 127-143.
    17. Chen, Xiaohong & Liao, Zhipeng & Sun, Yixiao, 2014. "Sieve inference on possibly misspecified semi-nonparametric time series models," Journal of Econometrics, Elsevier, vol. 178(P3), pages 639-658.
    18. Hall, Peter & Marron, J. S., 1987. "Estimation of integrated squared density derivatives," Statistics & Probability Letters, Elsevier, vol. 6(2), pages 109-115, November.
    19. Newey, Whitney K., 1994. "Kernel Estimation of Partial Means and a General Variance Estimator," Econometric Theory, Cambridge University Press, vol. 10(2), pages 1-21, June.
    20. Newey, Whitney K, 1990. "Semiparametric Efficiency Bounds," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 5(2), pages 99-135, April-Jun.
    21. Cho, Jin Seo & White, Halbert, 2018. "Directionally Differentiable Econometric Models," Econometric Theory, Cambridge University Press, vol. 34(5), pages 1101-1131, October.
    22. Richard Blundell & Xiaohong Chen & Dennis Kristensen, 2007. "Semi-Nonparametric IV Estimation of Shape-Invariant Engel Curves," Econometrica, Econometric Society, vol. 75(6), pages 1613-1669, November.
    23. Ai, Chunrong & Chen, Xiaohong, 2007. "Estimation of possibly misspecified semiparametric conditional moment restriction models with different conditioning variables," Journal of Econometrics, Elsevier, vol. 141(1), pages 5-43, November.
    24. Whitney K. Newey & James L. Powell, 2003. "Instrumental Variable Estimation of Nonparametric Models," Econometrica, Econometric Society, vol. 71(5), pages 1565-1578, September.
    25. Chunrong Ai & Xiaohong Chen, 2003. "Efficient Estimation of Models with Conditional Moment Restrictions Containing Unknown Functions," Econometrica, Econometric Society, vol. 71(6), pages 1795-1843, November.
    26. Xiaohong Chen & Xiaotong Shen, 1998. "Sieve Extremum Estimates for Weakly Dependent Data," Econometrica, Econometric Society, vol. 66(2), pages 289-314, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hidehiko Ichimura & Whitney K. Newey, 2022. "The influence function of semiparametric estimators," Quantitative Economics, Econometric Society, vol. 13(1), pages 29-61, January.
    2. Chen, Xiaohong & Pouzo, Demian & Powell, James L., 2019. "Penalized sieve GEL for weighted average derivatives of nonparametric quantile IV regressions," Journal of Econometrics, Elsevier, vol. 213(1), pages 30-53.
    3. Chen, Xiaohong & Pouzo, Demian, 2009. "Efficient estimation of semiparametric conditional moment models with possibly nonsmooth residuals," Journal of Econometrics, Elsevier, vol. 152(1), pages 46-60, September.
    4. Breunig, Christoph & Mammen, Enno & Simoni, Anna, 2018. "Nonparametric estimation in case of endogenous selection," Journal of Econometrics, Elsevier, vol. 202(2), pages 268-285.
    5. Victor Chernozhukov & Whitney Newey & Rahul Singh & Vasilis Syrgkanis, 2020. "Adversarial Estimation of Riesz Representers," Papers 2101.00009, arXiv.org, revised Apr 2024.
    6. Victor Chernozhukov & Juan Carlos Escanciano & Hidehiko Ichimura & Whitney K. Newey & James M. Robins, 2022. "Locally Robust Semiparametric Estimation," Econometrica, Econometric Society, vol. 90(4), pages 1501-1535, July.
    7. Liu, Lin & Mukherjee, Rajarshi & Robins, James M., 2024. "Assumption-lean falsification tests of rate double-robustness of double-machine-learning estimators," Journal of Econometrics, Elsevier, vol. 240(2).
    8. Andrew Bennett & Nathan Kallus & Xiaojie Mao & Whitney Newey & Vasilis Syrgkanis & Masatoshi Uehara, 2022. "Inference on Strongly Identified Functionals of Weakly Identified Functions," Papers 2208.08291, arXiv.org, revised Jun 2023.
    9. Escanciano, Juan Carlos & Li, Wei, 2021. "Optimal Linear Instrumental Variables Approximations," Journal of Econometrics, Elsevier, vol. 221(1), pages 223-246.
    10. Ai, Chunrong & Chen, Xiaohong, 2012. "The semiparametric efficiency bound for models of sequential moment restrictions containing unknown functions," Journal of Econometrics, Elsevier, vol. 170(2), pages 442-457.
    11. Jean-Jacques Forneron, 2019. "A Sieve-SMM Estimator for Dynamic Models," Papers 1902.01456, arXiv.org, revised Jan 2023.
    12. Victor Chernozhukov & Whitney K. Newey & Andres Santos, 2023. "Constrained Conditional Moment Restriction Models," Econometrica, Econometric Society, vol. 91(2), pages 709-736, March.
    13. Chen, Jiafeng & Chen, Xiaohong & Tamer, Elie, 2023. "Efficient estimation of average derivatives in NPIV models: Simulation comparisons of neural network estimators," Journal of Econometrics, Elsevier, vol. 235(2), pages 1848-1875.
    14. Chen, Xiaohong, 2007. "Large Sample Sieve Estimation of Semi-Nonparametric Models," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 76, Elsevier.
    15. Qi Li & Jeffrey Scott Racine, 2006. "Nonparametric Econometrics: Theory and Practice," Economics Books, Princeton University Press, edition 1, volume 1, number 8355.
    16. Zheng Fang & Juwon Seo, 2019. "A Projection Framework for Testing Shape Restrictions That Form Convex Cones," Papers 1910.07689, arXiv.org, revised Sep 2021.
    17. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    18. Babii, Andrii, 2020. "Honest Confidence Sets In Nonparametric Iv Regression And Other Ill-Posed Models," Econometric Theory, Cambridge University Press, vol. 36(4), pages 658-706, August.
    19. Xiaohong Chen & Demian Pouzo, 2012. "Estimation of Nonparametric Conditional Moment Models With Possibly Nonsmooth Generalized Residuals," Econometrica, Econometric Society, vol. 80(1), pages 277-321, January.
    20. Santos, Andres, 2011. "Instrumental variable methods for recovering continuous linear functionals," Journal of Econometrics, Elsevier, vol. 161(2), pages 129-146, April.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1712.07248. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.