IDEAS home Printed from https://ideas.repec.org/a/eee/econom/v146y2008i1p92-106.html
   My bibliography  Save this article

Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models

Author

Listed:
  • Kasahara, Hiroyuki
  • Shimotsu, Katsumi

Abstract

This paper analyzes the higher-order properties of the estimators based on the nested pseudo-likelihood (NPL) algorithm and the practical implementation of such estimators for parametric discrete Markov decision models. We derive the rate at which the NPL algorithm converges to the MLE and provide a theoretical explanation for the simulation results in Aguirregabiria and Mira [Aguirregabiria, V., Mira, P., 2002. Swapping the nested fixed point algorithm: A class of estimators for discrete Markov decision models. Econometrica 70, 1519-1543], in which iterating the NPL algorithm improves the accuracy of the estimator. We then propose a new NPL algorithm that can achieve quadratic convergence without fully solving the fixed point problem in every iteration and apply our estimation procedure to a finite mixture model. We also develop one-step NPL bootstrap procedures for discrete Markov decision models. The Monte Carlo simulation evidence based on a machine replacement model of Rust [Rust, J., 1987. Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher. Econometrica 55, 999-1033] shows that the proposed one-step bootstrap test statistics and confidence intervals improve upon the first order asymptotics even with a relatively small number of iterations.

Suggested Citation

  • Kasahara, Hiroyuki & Shimotsu, Katsumi, 2008. "Pseudo-likelihood estimation and bootstrap inference for structural discrete Markov decision models," Journal of Econometrics, Elsevier, vol. 146(1), pages 92-106, September.
  • Handle: RePEc:eee:econom:v:146:y:2008:i:1:p:92-106
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304-4076(08)00078-X
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Rust, John, 1987. "Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher," Econometrica, Econometric Society, vol. 55(5), pages 999-1033, September.
    2. Sumru Altuğ & Robert A. Miller, 1998. "The Effect of Work Experience on Female Wages and Labour Supply," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(1), pages 45-85.
    3. John Kennan & James R. Walker, 2011. "The Effect of Expected Income on Individual Migration Decisions," Econometrica, Econometric Society, vol. 79(1), pages 211-251, January.
    4. Andrews, Donald W.K., 2002. "EQUIVALENCE OF THE HIGHER ORDER ASYMPTOTIC EFFICIENCY OF k-STEP AND EXTREMUM STATISTICS," Econometric Theory, Cambridge University Press, vol. 18(5), pages 1040-1085, October.
    5. Hiroyuki Kasahara & Katsumi Shimotsu, 2006. "Nonparametric Identification And Estimation Of Finite Mixture Models Of Dynamic Discrete Choices," Working Paper 1092, Economics Department, Queen's University.
    6. Patrick Bajari & C. Lanier Benkard & Jonathan Levin, 2007. "Estimating Dynamic Models of Imperfect Competition," Econometrica, Econometric Society, vol. 75(5), pages 1331-1370, September.
    7. Victor Aguirregabiria & Pedro Mira, 2002. "Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models," Econometrica, Econometric Society, vol. 70(4), pages 1519-1543, July.
    8. Victor Aguirregabiria, 1999. "The Dynamics of Markups and Inventories in Retailing Firms," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(2), pages 275-308.
    9. V. Joseph Hotz & Robert A. Miller, 1993. "Conditional Choice Probabilities and the Estimation of Dynamic Models," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 60(3), pages 497-529.
    10. Ichimura, Hidehiko & Lee, Sokbae, 2010. "Characterization of the asymptotic distribution of semiparametric M-estimators," Journal of Econometrics, Elsevier, vol. 159(2), pages 252-266, December.
    11. Gonul, Fusun F., 1998. "Estimating price expectations in the OTC medicine market: An application of dynamic stochastic discrete choice models to scanner panel data," Journal of Econometrics, Elsevier, vol. 89(1-2), pages 41-56, November.
    12. Andrews,Donald W. K. & Stock,James H. (ed.), 2005. "Identification and Inference for Econometric Models," Cambridge Books, Cambridge University Press, number 9780521844413, September.
    13. Miller, Robert A. & Sanders, Seth G., 1997. "Human capital development and welfare participation," Carnegie-Rochester Conference Series on Public Policy, Elsevier, vol. 46(1), pages 1-43, June.
    14. Peter Arcidiacono, 2005. "Affirmative Action in Higher Education: How Do Admission and Financial Aid Rules Affect Future Earnings?," Econometrica, Econometric Society, vol. 73(5), pages 1477-1524, September.
    15. Paola Rota, 2004. "Estimating Labor Demand with Fixed Costs," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(1), pages 25-48, February.
    16. Donald W. K. Andrews, 2002. "Higher-Order Improvements of a Computationally Attractive "k"-Step Bootstrap for Extremum Estimators," Econometrica, Econometric Society, vol. 70(1), pages 119-162, January.
    17. John Rust & Christopher Phelan, 1997. "How Social Security and Medicare Affect Retirement Behavior in a World of Incomplete Markets," Econometrica, Econometric Society, vol. 65(4), pages 781-832, July.
    18. Namkee Ahn, 1995. "Measuring the Value of Children by Sex and Age Using a Dynamic Programming Model," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 62(3), pages 361-379.
    19. Robinson, Peter M, 1988. "The Stochastic Difference between Econometric Statistics," Econometrica, Econometric Society, vol. 56(3), pages 531-548, May.
    20. Miller, Robert A, 1984. "Job Matching and Occupational Choice," Journal of Political Economy, University of Chicago Press, vol. 92(6), pages 1086-1120, December.
    21. Kasahara, Hiroyuki & Lapham, Beverly, 2013. "Productivity and the decision to import and export: Theory and evidence," Journal of International Economics, Elsevier, vol. 89(2), pages 297-316.
    22. Arcidiacono, Peter, 2004. "Ability sorting and the returns to college major," Journal of Econometrics, Elsevier, vol. 121(1-2), pages 343-375.
    23. Pakes, Ariel S, 1986. "Patents as Options: Some Estimates of the Value of Holding European Patent Stocks," Econometrica, Econometric Society, vol. 54(4), pages 755-784, July.
    24. Davidson, Russell & MacKinnon, James G., 1999. "The Size Distortion Of Bootstrap Tests," Econometric Theory, Cambridge University Press, vol. 15(3), pages 361-376, June.
    25. Ariel Pakes & Michael Ostrovsky & Steven Berry, 2007. "Simple estimators for the parameters of discrete dynamic games (with entry/exit examples)," RAND Journal of Economics, RAND Corporation, vol. 38(2), pages 373-399, June.
    26. Arjan Heyma, 2004. "A structural dynamic analysis of retirement behaviour in the Netherlands," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 19(6), pages 739-759.
    27. John Haltiwanger & Russell Cooper & Laura Power, 1999. "Machine Replacement and the Business Cycle: Lumps and Bumps," American Economic Review, American Economic Association, vol. 89(4), pages 921-946, September.
    28. Margaret E. Slade & G.R.E.Q.A.M., 1998. "Optimal Pricing with Costly Adjustment: Evidence from Retail-Grocery Prices," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 65(1), pages 87-107.
    29. Gotz, Glenn A. & McCall, John J., 1980. "Estimation in sequential decisionmaking models : A methodological note," Economics Letters, Elsevier, vol. 6(2), pages 131-136.
    30. Davidson, Russell & MacKinnon, James G, 1999. "Bootstrap Testing in Nonlinear Models," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 40(2), pages 487-508, May.
    31. Rothwell, Geoffrey & Rust, John, 1997. "On the Optimal Lifetime of Nuclear Power Plants," Journal of Business & Economic Statistics, American Statistical Association, vol. 15(2), pages 195-208, April.
    32. Keane, Michael P & Wolpin, Kenneth I, 1997. "The Career Decisions of Young Men," Journal of Political Economy, University of Chicago Press, vol. 105(3), pages 473-522, June.
    33. Susumu Imai & Neelam Jain & Andrew Ching, 2009. "Bayesian Estimation of Dynamic Discrete Choice Models," Econometrica, Econometric Society, vol. 77(6), pages 1865-1899, November.
    34. Donna B. Gilleskie, 1998. "A Dynamic Stochastic Model of Medical Care Use and Work Absence," Econometrica, Econometric Society, vol. 66(1), pages 1-46, January.
    35. Wolpin, Kenneth I, 1984. "An Estimable Dynamic Stochastic Model of Fertility and Child Mortality," Journal of Political Economy, University of Chicago Press, vol. 92(5), pages 852-874, October.
    36. Patrick Bajari & Han Hong, 2006. "Semiparametric Estimation of a Dynamic Game of Incomplete Information," NBER Technical Working Papers 0320, National Bureau of Economic Research, Inc.
    37. Sanghamitra Das, 1992. "A Micro-Econometric Model of Capital Utilization and Retirement: The Case of the U.S. Cement Industry," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 59(2), pages 277-297.
    38. Susumu Imai & Michael P. Keane, 2004. "Intertemporal Labor Supply and Human Capital Accumulation," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 45(2), pages 601-641, May.
    39. Victor Aguirregabiria & Pedro Mira, 2007. "Sequential Estimation of Dynamic Discrete Games," Econometrica, Econometric Society, vol. 75(1), pages 1-53, January.
    40. Kennet, D Mark, 1994. "A Structural Model of Aircraft Engine Maintenance," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 9(4), pages 351-368, Oct.-Dec..
    41. Martin Pesendorfer & Philipp Schmidt-Dengler, 2008. "Asymptotic Least Squares Estimators for Dynamic Games -super-1," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 75(3), pages 901-928.
    42. Zvi Eckstein & Kenneth I. Wolpin, 1999. "Why Youths Drop Out of High School: The Impact of Preferences, Opportunities, and Abilities," Econometrica, Econometric Society, vol. 67(6), pages 1295-1340, November.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Srisuma, Sorawoot & Linton, Oliver, 2012. "Semiparametric estimation of Markov decision processes with continuous state space," Journal of Econometrics, Elsevier, vol. 166(2), pages 320-341.
    2. Victor Aguirregabiria & Victor Aguirregabiria & Aviv Nevo & Aviv Nevo, 2010. "Recent Developments in Empirical IO: Dynamic Demand and Dynamic Games," Working Papers tecipa-419, University of Toronto, Department of Economics.
    3. Karun Adusumilli & Dita Eckardt, 2019. "Temporal-Difference estimation of dynamic discrete choice models," Papers 1912.09509, arXiv.org, revised Dec 2022.
    4. Hiroyuki Kasahara & Katsumi Shimotsu, 2012. "Sequential Estimation of Structural Models With a Fixed Point Constraint," Econometrica, Econometric Society, vol. 80(5), pages 2303-2319, September.
    5. Victor Aguirregabiria & Arvind Magesan, 2013. "Euler Equations for the Estimation of Dynamic Discrete Choice Structural Models," Advances in Econometrics, in: Structural Econometric Models, volume 31, pages 3-44, Emerald Group Publishing Limited.
    6. Marmer, Vadim & Otsu, Taisuke, 2012. "Optimal comparison of misspecified moment restriction models under a chosen measure of fit," Journal of Econometrics, Elsevier, vol. 170(2), pages 538-550.
    7. Federico A Bugni & Jackson Bunting, 2021. "On the Iterated Estimation of Dynamic Discrete Choice Games [Pseudo maximum likelihood estimation of structural models involving fixed-point problems]," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 88(3), pages 1031-1073.
    8. Adam Dearing & Jason R. Blevins, 2019. "Efficient and Convergent Sequential Pseudo-Likelihood Estimation of Dynamic Discrete Games," Papers 1912.10488, arXiv.org, revised Apr 2024.
    9. Hiroyuki Kasahara & Katsumi Shimotsu, 2018. "Estimation of Discrete Choice Dynamic Programming Models," The Japanese Economic Review, Springer, vol. 69(1), pages 28-58, March.
    10. Aguirregabiria, Victor & Magesan, Arvind, 2013. "Euler Equations for the Estimation of Dynamic Discrete Choice Structural," MPRA Paper 46056, University Library of Munich, Germany.
    11. Murasawa, Yasutomo, 2023. "大学中退の逐次意思決定モデルの構造推定 [Structural estimation of a sequential decision model of college dropout]," MPRA Paper 118183, University Library of Munich, Germany.
    12. Aguirregabiria, Victor & Ho, Chun-Yu, 2012. "A dynamic oligopoly game of the US airline industry: Estimation and policy experiments," Journal of Econometrics, Elsevier, vol. 168(1), pages 156-173.
    13. An, Yonghong & Hu, Yingyao & Xiao, Ruli, 2021. "Dynamic decisions under subjective expectations: A structural analysis," Journal of Econometrics, Elsevier, vol. 222(1), pages 645-675.
    14. Komarova, Tatiana & Sanches, Fábio Adriano & Silva Junior, Daniel & Srisuma, Sorawoot, 2018. "Joint analysis of the discount factor and payoff parameters in dynamic discrete choice games," LSE Research Online Documents on Economics 86858, London School of Economics and Political Science, LSE Library.
    15. Hidehiko Ichimura, 2018. "Announcement," The Japanese Economic Review, Springer, vol. 69(1), pages 26-27, March.
    16. Yu Zheng & Juan Pantano, 2012. "Using Subjective Expectations Data to Allow for Unobserved Heterogeneity in Hotz-Miller Estimation Strategies," 2012 Meeting Papers 940, Society for Economic Dynamics.
    17. Gupta, Abhimanyu, 2023. "Efficient closed-form estimation of large spatial autoregressions," Journal of Econometrics, Elsevier, vol. 232(1), pages 148-167.
    18. Aguirregabiria, Victor, 2009. "Estimation of Dynamic Discrete Games Using the Nested Pseudo Likelihood Algorithm: Code and Application," MPRA Paper 17329, University Library of Munich, Germany.
    19. Fabio A. Miessi Sanches & Daniel Silva Junior, Sorawoot Srisuma, 2014. "Ordinary Least Squares Estimation for a Dynamic Game," Working Papers, Department of Economics 2014_19, University of São Paulo (FEA-USP), revised 23 Feb 2015.
    20. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aguirregabiria, Victor & Mira, Pedro, 2010. "Dynamic discrete choice structural models: A survey," Journal of Econometrics, Elsevier, vol. 156(1), pages 38-67, May.
    2. Hiroyuki Kasahara & Katsumi Shimotsu, 2006. "Nested Pseudo-likelihood Estimation and Bootstrap-based Inference for Structural Discrete Markov Decision Models," University of Western Ontario, Departmental Research Report Series 20064, University of Western Ontario, Department of Economics.
    3. Peter Arcidiacono & Robert A. Miller, 2011. "Conditional Choice Probability Estimation of Dynamic Discrete Choice Models With Unobserved Heterogeneity," Econometrica, Econometric Society, vol. 79(6), pages 1823-1867, November.
    4. Hiroyuki Kasahara & Katsumi Shimotsu, 2018. "Estimation of Discrete Choice Dynamic Programming Models," The Japanese Economic Review, Japanese Economic Association, vol. 69(1), pages 28-58, March.
    5. Victor Aguirregabiria, 2006. "Another Look at the Identification of Dynamic Discrete Decision Processes: With an Application to Retirement Behavior," 2006 Meeting Papers 169, Society for Economic Dynamics.
    6. Peter Arcidiacono & Paul B. Ellickson, 2011. "Practical Methods for Estimation of Dynamic Discrete Choice Models," Annual Review of Economics, Annual Reviews, vol. 3(1), pages 363-394, September.
    7. Victor Aguirregabiria & Margaret Slade, 2017. "Empirical models of firms and industries," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 50(5), pages 1445-1488, December.
    8. Le-Yu Chen, 2009. "Identification of structural dynamic discrete choice models," CeMMAP working papers CWP08/09, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    9. Sebastian Galiani & Juan Pantano, 2021. "Structural Models: Inception and Frontier," NBER Working Papers 28698, National Bureau of Economic Research, Inc.
    10. Arcidiacono, Peter & Miller, Robert A., 2020. "Identifying dynamic discrete choice models off short panels," Journal of Econometrics, Elsevier, vol. 215(2), pages 473-485.
    11. Peter Arcidiacono, Holger Sieg, Frank Sloan, 2001. "Living Rationally Under the Volcano? Heavy Drinking and Smoking Among the Elderly," Computing in Economics and Finance 2001 207, Society for Computational Economics.
    12. Keane, Michael P. & Todd, Petra E. & Wolpin, Kenneth I., 2011. "The Structural Estimation of Behavioral Models: Discrete Choice Dynamic Programming Methods and Applications," Handbook of Labor Economics, in: O. Ashenfelter & D. Card (ed.), Handbook of Labor Economics, edition 1, volume 4, chapter 4, pages 331-461, Elsevier.
    13. Victor Aguirregabiria, 2005. "Another Look at the Identification of Dynamic Discrete Decision Processes," Econometrics 0504006, University Library of Munich, Germany.
    14. Hanming Fang & Yang Wang, 2015. "Estimating Dynamic Discrete Choice Models With Hyperbolic Discounting, With An Application To Mammography Decisions," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 56(2), pages 565-596, May.
    15. Kalouptsidi, Myrto & Scott, Paul & Souza-Rodrigues, Edouardo, 2015. "Identification of Counterfactuals and Payoffs in Dynamic Discrete Choice with an Application to Land Use," TSE Working Papers 15-596, Toulouse School of Economics (TSE).
    16. Myrto Kalouptsidi & Paul T. Scott & Eduardo Souza-Rodrigues, 2015. "Identification of Counterfactuals in Dynamic Discrete Choice Models," NBER Working Papers 21527, National Bureau of Economic Research, Inc.
    17. Hu Yingyao & Shum Matthew & Tan Wei & Xiao Ruli, 2017. "A Simple Estimator for Dynamic Models with Serially Correlated Unobservables," Journal of Econometric Methods, De Gruyter, vol. 6(1), pages 1-16, January.
    18. Hu, Yingyao & Shum, Matthew, 2012. "Nonparametric identification of dynamic models with unobserved state variables," Journal of Econometrics, Elsevier, vol. 171(1), pages 32-44.
    19. Peter Arcidiacono & Holger Sieg & Frank Sloan, 2007. "Living Rationally Under The Volcano? An Empirical Analysis Of Heavy Drinking And Smoking," International Economic Review, Department of Economics, University of Pennsylvania and Osaka University Institute of Social and Economic Research Association, vol. 48(1), pages 37-65, February.
    20. Hiroyuki Kasahara & Katsumi Shimotsu, 2012. "Sequential Estimation of Structural Models With a Fixed Point Constraint," Econometrica, Econometric Society, vol. 80(5), pages 2303-2319, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:econom:v:146:y:2008:i:1:p:92-106. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/jeconom .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.