IDEAS home Printed from https://ideas.repec.org/a/bla/popmgt/v31y2022i12p4558-4571.html
   My bibliography  Save this article

On the causality and plausibility of treatment effects in operations management research

Author

Listed:
  • Sunil Mithas
  • Yanzhen Chen
  • Yatang Lin
  • Alysson De Oliveira Silveira

Abstract

Empirical research in operations management (OM) has made rapid strides in the last 30 years, and increasingly, OM researchers are leveraging methods used in the econometrics and statistics literature to assess the causal effects of interventions. We discuss the two key challenges in assessing causality with observational data (i.e., baseline bias, differential treatment effect bias) and how dominant identification approaches such as matching, instrumental variables, regression discontinuity, difference‐in‐differences, and fixed effects deal with such challenges. We surface the key underlying assumptions of different causal estimation methods and discuss how OM scholars have used these methods in the last few years. We hope that reflecting on the plausibility and substantive meaning of underlying assumptions regarding different identification strategies in a particular context will lead to a better conceptualization, execution, evaluation, dissemination, and consumption of OM research. We conclude with a few thoughts that authors and reviewers may find helpful in their research as they engage in discourse related to causality.

Suggested Citation

  • Sunil Mithas & Yanzhen Chen & Yatang Lin & Alysson De Oliveira Silveira, 2022. "On the causality and plausibility of treatment effects in operations management research," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4558-4571, December.
  • Handle: RePEc:bla:popmgt:v:31:y:2022:i:12:p:4558-4571
    DOI: 10.1111/poms.13863
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/poms.13863
    Download Restriction: no

    File URL: https://libkey.io/10.1111/poms.13863?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jorge Mejia & Gloria Urrea & Alfonso J. Pedraza‐Martinez, 2019. "Operational Transparency on Crowdfunding Platforms: Effect on Donations for Emergency Response," Production and Operations Management, Production and Operations Management Society, vol. 28(7), pages 1773-1791, July.
    2. Keane, Michael P., 2010. "Structural vs. atheoretic approaches to econometrics," Journal of Econometrics, Elsevier, vol. 156(1), pages 3-20, May.
    3. Angus Deaton, 2010. "Instruments, Randomization, and Learning about Development," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 424-455, June.
    4. Sulin Ba & Shu He & Shun‐Yang Lee, 2022. "Mobile App Adoption and Its Differential Impact on Consumer Shopping Behavior," Production and Operations Management, Production and Operations Management Society, vol. 31(2), pages 764-780, February.
    5. Dmitry Arkhangelsky & Susan Athey & David A. Hirshberg & Guido W. Imbens & Stefan Wager, 2021. "Synthetic Difference-in-Differences," American Economic Review, American Economic Association, vol. 111(12), pages 4088-4118, December.
    6. Hainmueller, Jens, 2012. "Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies," Political Analysis, Cambridge University Press, vol. 20(1), pages 25-46, January.
    7. Guo Li & Na Li & Suresh P. Sethi, 2021. "Does CSR Reduce Idiosyncratic Risk? Roles of Operational Efficiency and AI Innovation," Production and Operations Management, Production and Operations Management Society, vol. 30(7), pages 2027-2045, July.
    8. Gow, Ian D. & Larcker, David F. & Reiss, Peter C., 2016. "Causal Inference in Accounting Research," Research Papers 3393, Stanford University, Graduate School of Business.
    9. Imai, Kosuke & Kim, In Song, 2021. "On the Use of Two-Way Fixed Effects Regression Models for Causal Inference with Panel Data," Political Analysis, Cambridge University Press, vol. 29(3), pages 405-415, July.
    10. Gruber, Jonathan, 1994. "The Incidence of Mandated Maternity Benefits," American Economic Review, American Economic Association, vol. 84(3), pages 622-641, June.
    11. Guido W. Imbens, 2010. "Better LATE Than Nothing: Some Comments on Deaton (2009) and Heckman and Urzua (2009)," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 399-423, June.
    12. Elizabeth U. Cascio & Ethan G. Lewis, 2006. "Schooling and the Armed Forces Qualifying Test: Evidence from School-Entry Laws," Journal of Human Resources, University of Wisconsin Press, vol. 41(2).
    13. Betz, Timm & Cook, Scott J. & Hollenbach, Florian M., 2018. "On the Use and Abuse of Spatial Instruments," Political Analysis, Cambridge University Press, vol. 26(4), pages 474-479, October.
    14. Xingyue Zhang & Yuliang Yao, 2020. "How Much is Too Much? The Effect of Offline Call Intensity on Online Purchase of Digital Services," Production and Operations Management, Production and Operations Management Society, vol. 29(3), pages 509-525, March.
    15. Howard Hao-Chun Chuang & Rogelio Oliva & Olga Perdikaki, 2016. "Traffic-Based Labor Planning in Retail Stores," Production and Operations Management, Production and Operations Management Society, vol. 25(1), pages 96-113, January.
    16. Eduardo Cavallo & Sebastian Galiani & Ilan Noy & Juan Pantano, 2013. "Catastrophic Natural Disasters and Economic Growth," The Review of Economics and Statistics, MIT Press, vol. 95(5), pages 1549-1561, December.
    17. Berry, Steven & Levinsohn, James & Pakes, Ariel, 1995. "Automobile Prices in Market Equilibrium," Econometrica, Econometric Society, vol. 63(4), pages 841-890, July.
    18. King, Gary & Nielsen, Richard, 2019. "Why Propensity Scores Should Not Be Used for Matching," Political Analysis, Cambridge University Press, vol. 27(4), pages 435-454, October.
    19. Randy V. Bradley & Terry L. Esper & Joonhwan In & Kang B. Lee & Bogdan C. Bichescu & Terry Anthony Byrd, 2018. "The Joint Use of RFID and EDI: Implications for Hospital Performance," Production and Operations Management, Production and Operations Management Society, vol. 27(11), pages 2071-2090, November.
    20. Alberto Abadie & Alexis Diamond & Jens Hainmueller, 2015. "Comparative Politics and the Synthetic Control Method," American Journal of Political Science, John Wiley & Sons, vol. 59(2), pages 495-510, February.
    21. Turgay Ayer & Mehmet U. S. Ayvaci & Zeynal Karaca & Jan Vlachy, 2019. "The Impact of Health Information Exchanges on Emergency Department Length of Stay," Production and Operations Management, Production and Operations Management Society, vol. 28(3), pages 740-758, March.
    22. Alberto Abadie & Javier Gardeazabal, 2003. "The Economic Costs of Conflict: A Case Study of the Basque Country," American Economic Review, American Economic Association, vol. 93(1), pages 113-132, March.
    23. Ian D. Gow & David F. Larcker & Peter C. Reiss, 2016. "Causal Inference in Accounting Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 477-523, May.
    24. Sunil Mithas & M. S. Krishnan, 2009. "From Association to Causation via a Potential Outcomes Approach," Information Systems Research, INFORMS, vol. 20(2), pages 295-313, June.
    25. Christian Terwiesch & Marcelo Olivares & Bradley R. Staats & Vishal Gaur, 2020. "OM Forum—A Review of Empirical Operations Management over the Last Two Decades," Manufacturing & Service Operations Management, INFORMS, vol. 22(4), pages 656-668, July.
    26. Hsihui Chang & Sheng Liu & Raj Mashruwala, 2022. "Customer Bargaining Power, Strategic Fit, and Supplier Performance," Production and Operations Management, Production and Operations Management Society, vol. 31(4), pages 1492-1509, April.
    27. Sun, Liyang & Abraham, Sarah, 2021. "Estimating dynamic treatment effects in event studies with heterogeneous treatment effects," Journal of Econometrics, Elsevier, vol. 225(2), pages 175-199.
    28. Xinshu Zhao & John G. Lynch & Qimei Chen, 2010. "Reconsidering Baron and Kenny: Myths and Truths about Mediation Analysis," Journal of Consumer Research, Journal of Consumer Research Inc., vol. 37(2), pages 197-206, August.
    29. Terence J. V. Saldanha & Dongwon Lee & Sunil Mithas, 2020. "Aligning Information Technology and Business: The Differential Effects of Alignment During Investment Planning, Delivery, and Change," Information Systems Research, INFORMS, vol. 31(4), pages 1260-1281, December.
    30. Hahn, Jinyong & Todd, Petra & Van der Klaauw, Wilbert, 2001. "Identification and Estimation of Treatment Effects with a Regression-Discontinuity Design," Econometrica, Econometric Society, vol. 69(1), pages 201-209, January.
    31. Deaton, Angus & Cartwright, Nancy, 2018. "Understanding and misunderstanding randomized controlled trials," Social Science & Medicine, Elsevier, vol. 210(C), pages 2-21.
    32. Joshua D. Angrist & Jörn-Steffen Pischke, 2009. "Mostly Harmless Econometrics: An Empiricist's Companion," Economics Books, Princeton University Press, edition 1, number 8769.
    33. Michelle Marcus & Pedro H. C. Sant’Anna, 2021. "The Role of Parallel Trends in Event Study Settings: An Application to Environmental Economics," Journal of the Association of Environmental and Resource Economists, University of Chicago Press, vol. 8(2), pages 235-275.
    34. Emmanouil Avgerinos & Bilal Gokpinar, 2018. "Task Variety in Professional Service Work: When It Helps and When It Hurts," Production and Operations Management, Production and Operations Management Society, vol. 27(7), pages 1368-1389, July.
    35. Stock, James H & Wright, Jonathan H & Yogo, Motohiro, 2002. "A Survey of Weak Instruments and Weak Identification in Generalized Method of Moments," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(4), pages 518-529, October.
    36. Sunil Mithas & Zhi‐Long Chen & Terence J.V. Saldanha & Alysson De Oliveira Silveira, 2022. "How will artificial intelligence and Industry 4.0 emerging technologies transform operations management?," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4475-4487, December.
    37. Aleda Roth & Eve Rosenzweig, 2020. "Advancing Empirical Science in Operations Management Research: A Clarion Call to Action," Manufacturing & Service Operations Management, INFORMS, vol. 22(1), pages 179-190, January.
    38. Sina Golara & Kevin J. Dooley & Nasim Mousavi, 2021. "Are Dealers Still Relevant? How Dealer Service Quality Impacts Manufacturer Success," Production and Operations Management, Production and Operations Management Society, vol. 30(10), pages 3560-3578, October.
    39. Pratt, John W. & Schlaifer, Robert, 1988. "On the interpretation and observation of laws," Journal of Econometrics, Elsevier, vol. 39(1-2), pages 23-52.
    40. Abadie, Alberto & Diamond, Alexis & Hainmueller, Jens, 2010. "Synthetic Control Methods for Comparative Case Studies: Estimating the Effect of California’s Tobacco Control Program," Journal of the American Statistical Association, American Statistical Association, vol. 105(490), pages 493-505.
    41. Erin C. McKie & Mark E. Ferguson & Michael R. Galbreth & Sriram Venkataraman, 2018. "How Do Consumers Choose between Multiple Product Generations and Conditions? An Empirical Study of iPad Sales on eBay," Production and Operations Management, Production and Operations Management Society, vol. 27(8), pages 1574-1594, August.
    42. James Heckman, 1997. "Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations," Journal of Human Resources, University of Wisconsin Press, vol. 32(3), pages 441-462.
    43. Steven A. Melnyk & Barbara B. Flynn & Amrou Awaysheh, 2018. "The best of times and the worst of times: empirical operations and supply chain management research," International Journal of Production Research, Taylor & Francis Journals, vol. 56(1-2), pages 164-192, January.
    44. Ling Xue & Peijian Song & Arun Rai & Cheng Zhang & Xia Zhao, 2019. "Implications of Application Programming Interfaces for Third‐Party New App Development and Copycatting," Production and Operations Management, Production and Operations Management Society, vol. 28(8), pages 1887-1902, August.
    45. Stanley Frederick W. T. Lim & Elliot Rabinovich & Sungho Park & Minha Hwang, 2021. "Shopping Activity at Warehouse Club Stores and Its Competitive and Network Density Implications," Production and Operations Management, Production and Operations Management Society, vol. 30(1), pages 28-46, January.
    46. Matias D. Cattaneo & Michael Jansson & Xinwei Ma, 2018. "Manipulation testing based on density discontinuity," Stata Journal, StataCorp LP, vol. 18(1), pages 234-261, March.
    47. Kosuke Imai & In Song Kim, 2019. "When Should We Use Unit Fixed Effects Regression Models for Causal Inference with Longitudinal Data?," American Journal of Political Science, John Wiley & Sons, vol. 63(2), pages 467-490, April.
    48. Joshua D. Angrist & Victor Lavy, 1999. "Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(2), pages 533-575.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Subodha Kumar & Christopher S. Tang, 2022. "Expanding the boundaries of the discipline: The 30th‐anniversary issue of Production and Operations Management," Production and Operations Management, Production and Operations Management Society, vol. 31(12), pages 4257-4261, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Peter Hull & Michal Kolesár & Christopher Walters, 2022. "Labor by design: contributions of David Card, Joshua Angrist, and Guido Imbens," Scandinavian Journal of Economics, Wiley Blackwell, vol. 124(3), pages 603-645, July.
    2. Roth, Jonathan & Sant’Anna, Pedro H.C. & Bilinski, Alyssa & Poe, John, 2023. "What’s trending in difference-in-differences? A synthesis of the recent econometrics literature," Journal of Econometrics, Elsevier, vol. 235(2), pages 2218-2244.
    3. Susan Athey & Guido W. Imbens, 2017. "The State of Applied Econometrics: Causality and Policy Evaluation," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 3-32, Spring.
    4. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    5. Guido W. Imbens, 2020. "Potential Outcome and Directed Acyclic Graph Approaches to Causality: Relevance for Empirical Practice in Economics," Journal of Economic Literature, American Economic Association, vol. 58(4), pages 1129-1179, December.
    6. Nicolas R. Ziebarth, 2018. "Social Insurance and Health," Contributions to Economic Analysis, in: Health Econometrics, volume 127, pages 57-84, Emerald Group Publishing Limited.
    7. Dmitry Arkhangelsky & Guido Imbens, 2023. "Causal Models for Longitudinal and Panel Data: A Survey," Papers 2311.15458, arXiv.org, revised Jun 2024.
    8. Huber, Martin, 2019. "An introduction to flexible methods for policy evaluation," FSES Working Papers 504, Faculty of Economics and Social Sciences, University of Freiburg/Fribourg Switzerland.
    9. Committee, Nobel Prize, 2021. "Answering causal questions using observational data," Nobel Prize in Economics documents 2021-2, Nobel Prize Committee.
    10. Christian Leuz & Peter D. Wysocki, 2016. "The Economics of Disclosure and Financial Reporting Regulation: Evidence and Suggestions for Future Research," Journal of Accounting Research, Wiley Blackwell, vol. 54(2), pages 525-622, May.
    11. Nicolaj N. Mühlbach, 2020. "Tree-based Synthetic Control Methods: Consequences of moving the US Embassy," CREATES Research Papers 2020-04, Department of Economics and Business Economics, Aarhus University.
    12. Dennis Shen & Peng Ding & Jasjeet Sekhon & Bin Yu, 2022. "Same Root Different Leaves: Time Series and Cross-Sectional Methods in Panel Data," Papers 2207.14481, arXiv.org, revised Oct 2022.
    13. Pekka Malo & Juha Eskelinen & Xun Zhou & Timo Kuosmanen, 2024. "Computing Synthetic Controls Using Bilevel Optimization," Computational Economics, Springer;Society for Computational Economics, vol. 64(2), pages 1113-1136, August.
    14. Hao, Shiming, 2021. "True structure change, spurious treatment effect? A novel approach to disentangle treatment effects from structure changes," MPRA Paper 108679, University Library of Munich, Germany.
    15. Cummins Joseph & Miller Douglas L. & Smith Brock & Simon David, 2024. "Matching on Noise: Finite Sample Bias in the Synthetic Control Estimator," Journal of Econometric Methods, De Gruyter, vol. 13(1), pages 67-95, January.
    16. Guido W. Imbens, 2022. "Causality in Econometrics: Choice vs Chance," Econometrica, Econometric Society, vol. 90(6), pages 2541-2566, November.
    17. Denis Fougère & Nicolas Jacquemet, 2020. "Policy Evaluation Using Causal Inference Methods," SciencePo Working papers Main hal-03455978, HAL.
    18. Mauricio Villamizar‐Villegas & Freddy A. Pinzon‐Puerto & Maria Alejandra Ruiz‐Sanchez, 2022. "A comprehensive history of regression discontinuity designs: An empirical survey of the last 60 years," Journal of Economic Surveys, Wiley Blackwell, vol. 36(4), pages 1130-1178, September.
    19. Olson, Adam J. & Yust, Christopher G. & Christensen, Brant E., 2023. "Are public health policies associated with corporate innovation? Evidence from U.S. nonsmoking laws," Research Policy, Elsevier, vol. 52(10).
    20. Eli Ben‐Michael & Avi Feller & Jesse Rothstein, 2022. "Synthetic controls with staggered adoption," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(2), pages 351-381, April.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:popmgt:v:31:y:2022:i:12:p:4558-4571. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://onlinelibrary.wiley.com/journal/10.1111/(ISSN)1937-5956 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.