IDEAS home Printed from https://ideas.repec.org/a/eee/bushor/v65y2022i4p481-492.html
   My bibliography  Save this article

The perils of working with big data, and a SMALL checklist you can use to recognize them

Author

Listed:
  • Brave, Scott A.
  • Butters, R. Andrew
  • Fogarty, Michael

Abstract

The use of big data to help explain fluctuations in the broader economy and key business performance indicators is now so commonplace that in some instances it has even begun to rival more traditional measures. Big data sources can very often provide advantages when compared with these more traditional data sources, but with these advantages also come potential pitfalls. We lay out a checklist called SMALL that we have developed in order to help interested parties as they navigate the big data minefield. Based on a set of five questions, the SMALL checklist should help users of big data draw justifiable conclusions and avoid making mistakes in matters of interpretation. To demonstrate, we provide several case studies that demonstrate the subtle nuances of several of these new big data sets and show how the problems they face often closely relate to age-old concerns that more traditional data sources are also forced to tackle.

Suggested Citation

  • Brave, Scott A. & Butters, R. Andrew & Fogarty, Michael, 2022. "The perils of working with big data, and a SMALL checklist you can use to recognize them," Business Horizons, Elsevier, vol. 65(4), pages 481-492.
    • Handle: RePEc:eee:bushor:v:65:y:2022:i:4:p:481-492
    DOI: 10.1016/j.bushor.2021.06.004
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0007681321001178
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.bushor.2021.06.004?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Tucker McElroy, 2017. "Multivariate Seasonal Adjustment, Economic Identities, and Seasonal Taxonomy," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 35(4), pages 611-625, October.
    2. Alharthi, Abdulkhaliq & Krotov, Vlad & Bowman, Michael, 2017. "Addressing barriers to big data," Business Horizons, Elsevier, vol. 60(3), pages 285-292.
    3. Alberto Cavallo, 2017. "Are Online and Offline Prices Similar? Evidence from Large Multi-channel Retailers," American Economic Review, American Economic Association, vol. 107(1), pages 283-303, January.
    4. Pierre Azoulay & Joshua S Graff Zivin & Danielle Li & Bhaven N Sampat, 2019. "Public R&D Investments and Private-sector Patenting: Evidence from NIH Funding Rules," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 86(1), pages 117-152.
    5. Lee, In, 2017. "Big data: Dimensions, evolution, impacts, and challenges," Business Horizons, Elsevier, vol. 60(3), pages 293-303.
    6. Tyler Atkinson & Jim Dolmas & Christoffer Koch & Evan F. Koenig & Karel Mertens & Anthony Murphy & Kei-Mu Yi, 2020. "Mobility and Engagement Following the SARS-Cov-2 Outbreak," Working Papers 2014, Federal Reserve Bank of Dallas.
    7. Sumedha Gupta & Thuy D. Nguyen & Felipe Lozano Rojas & Shyam Raman & Byungkyu Lee & Ana Bento & Kosali I. Simon & Coady Wing, 2020. "Tracking Public and Private Responses to the COVID-19 Epidemic: Evidence from State and Local Government Actions," NBER Working Papers 27027, National Bureau of Economic Research, Inc.
    8. Barsky, Robert B & Miron, Jeffrey A, 1989. "The Seasonal Cycle and the Business Cycle," Journal of Political Economy, University of Chicago Press, vol. 97(3), pages 503-534, June.
    9. Crane, Leland D. & Decker, Ryan A. & Flaaen, Aaron & Hamins-Puertolas, Adrian & Kurz, Christopher, 2022. "Business exit during the COVID-19 pandemic: Non-traditional measures in historical context," Journal of Macroeconomics, Elsevier, vol. 72(C).
    10. Aditya Aladangady & Shifrah Aron-Dine & Wendy Dunn & Laura Feiveson & Paul Lengermann & Claudia Sahm, 2021. "From Transaction Data to Economic Statistics: Constructing Real-Time, High-Frequency, Geographic Measures of Consumer Spending," NBER Chapters, in: Big Data for Twenty-First-Century Economic Statistics, pages 115-145, National Bureau of Economic Research, Inc.
    11. Olley, G Steven & Pakes, Ariel, 1996. "The Dynamics of Productivity in the Telecommunications Equipment Industry," Econometrica, Econometric Society, vol. 64(6), pages 1263-1297, November.
    12. Jose Maria Barrero & Nick Bloom & Steven J. Davis, 2020. "60 Million Fewer Commuting Hours Per Day: How Americans Use Time Saved by Working from Home," Working Papers 2020-132, Becker Friedman Institute for Research In Economics.
    13. Earley, Christine E., 2015. "Data analytics in auditing: Opportunities and challenges," Business Horizons, Elsevier, vol. 58(5), pages 493-500.
    14. Alberto Cavallo & Roberto Rigobon, 2016. "The Billion Prices Project: Using Online Prices for Measurement and Research," Journal of Economic Perspectives, American Economic Association, vol. 30(2), pages 151-178, Spring.
    15. Susan Herbst-Murphy, 2013. "Clearing and settlement of interbank card transactions: a MasterCard tutorial for Federal Reserve payments analysts," Consumer Finance Institute discussion papers 13-02, Federal Reserve Bank of Philadelphia.
    16. Menelik Geremew & François Gourio, 2018. "Seasonal and Business Cycles of U.S. Employment," Economic Perspectives, Federal Reserve Bank of Chicago, issue 3, pages 1-28.
    17. Alexander W. Bartik & Marianne Bertrand & Feng Lin & Jesse Rothstein & Matthew Unrath, 2020. "Measuring the Labor Market at the Onset of the COVID-19 Crisis," Brookings Papers on Economic Activity, Economic Studies Program, The Brookings Institution, vol. 51(2 (Summer), pages 239-268;316.
    18. Tomaz Cajner & Leland D. Crane & Ryan A. Decker & Adrian Hamins-Puertolas & Christopher J. Kurz & Tyler Radler, 2018. "Using Payroll Processor Microdata to Measure Aggregate Labor Market Activity," Finance and Economics Discussion Series 2018-005, Board of Governors of the Federal Reserve System (U.S.).
    19. Judith A. Chevalier & Anil K. Kashyap & Peter E. Rossi, 2003. "Why Don't Prices Rise During Periods of Peak Demand? Evidence from Scanner Data," American Economic Review, American Economic Association, vol. 93(1), pages 15-37, March.
    20. Nagle, Tadhg & Redman, Tom & Sammon, David, 2020. "Assessing data quality: A managerial call to action," Business Horizons, Elsevier, vol. 63(3), pages 325-337.
    21. Walker, Russell, 2015. "From Big Data to Big Profits: Success with Data and Analytics," OUP Catalogue, Oxford University Press, number 9780199378326.
    22. Tomaz Cajner & Leland D. Crane & Ryan A. Decker & Adrian Hamins-Puertolas & Christopher J. Kurz, 2020. "Tracking Labor Market Developments during the COVID-19 Pandemic: A Preliminary Assessment," Finance and Economics Discussion Series 2020-030, Board of Governors of the Federal Reserve System (U.S.).
    23. Pham, Xuan & Stack, Martin, 2018. "How data analytics is transforming agriculture," Business Horizons, Elsevier, vol. 61(1), pages 125-133.
    24. R. Andrew Butters, 2020. "Demand Volatility, Adjustment Costs, and Productivity: An Examination of Capacity Utilization in Hotels and Airlines," American Economic Journal: Microeconomics, American Economic Association, vol. 12(4), pages 1-44, November.
    25. Jaap H. Abbring & Jeffrey R. Campbell, 2005. "A Firm's First Year," Tinbergen Institute Discussion Papers 05-046/3, Tinbergen Institute.
    26. Vanhoof Maarten & Reis Fernando & Ploetz Thomas & Smoreda Zbigniew, 2018. "Assessing the Quality of Home Detection from Mobile Phone Data for Official Statistics," Journal of Official Statistics, Sciendo, vol. 34(4), pages 935-960, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Crane, Leland D. & Decker, Ryan A. & Flaaen, Aaron & Hamins-Puertolas, Adrian & Kurz, Christopher, 2022. "Business exit during the COVID-19 pandemic: Non-traditional measures in historical context," Journal of Macroeconomics, Elsevier, vol. 72(C).
    2. Ademmer, Martin & Beckmann, Joscha & Bode, Eckhardt & Boysen-Hogrefe, Jens & Funke, Manuel & Hauber, Philipp & Heidland, Tobias & Hinz, Julian & Jannsen, Nils & Kooths, Stefan & Söder, Mareike & Stame, 2021. "Big Data in der makroökonomischen Analyse," Kieler Beiträge zur Wirtschaftspolitik 32, Kiel Institute for the World Economy (IfW Kiel).
    3. Shun-Yang Lee & Julian Runge & Daniel Yoo & Yakov Bart & Anett Gyurak & J. W. Schneider, 2023. "COVID-19 Demand Shocks Revisited: Did Advertising Technology Help Mitigate Adverse Consequences for Small and Midsize Businesses?," Papers 2307.09035, arXiv.org, revised Jan 2024.
    4. Patrick Bajari & Zhihao Cen & Victor Chernozhukov & Manoj Manukonda & Jin Wang & Ramon Huerta & Junbo Li & Ling Leng & George Monokroussos & Suhas Vijaykunar & Shan Wan, 2023. "Hedonic prices and quality adjusted price indices powered by AI," CeMMAP working papers 08/23, Institute for Fiscal Studies.
    5. Laura Birg & Anna Goeddeke, 2016. "Christmas Economics—A Sleigh Ride," Economic Inquiry, Western Economic Association International, vol. 54(4), pages 1980-1984, October.
    6. Francis Aboagye‐Otchere & Cletus Agyenim‐Boateng & Abdulai Enusah & Theodora Ekua Aryee, 2021. "A Review of Big Data Research in Accounting," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 28(4), pages 268-283, October.
    7. Umit Can & Bilal Alatas, 2017. "Big Social Network Data and Sustainable Economic Development," Sustainability, MDPI, vol. 9(11), pages 1-19, November.
    8. Cowan, Benjamin & Jones, Todd R. & Swigert, Jeffrey, 2024. "Parental and Student Time Use Around the Academic Year," Journal of Economic Behavior & Organization, Elsevier, vol. 224(C), pages 66-110.
    9. Argente, David & Lee, Munseob & Moreira, Sara, 2018. "Innovation and product reallocation in the great recession," Journal of Monetary Economics, Elsevier, vol. 93(C), pages 1-20.
    10. Brian Fabo & Sharon Sarah Belli, 2017. "(Un)beliveable wages? An analysis of minimum wage policies in Europe from a living wage perspective," IZA Journal of Labor Policy, Springer;Forschungsinstitut zur Zukunft der Arbeit GmbH (IZA), vol. 6(1), pages 1-11, December.
    11. David Staines, 2023. "Stochastic Equilibrium the Lucas Critique and Keynesian Economics," Papers 2312.16214, arXiv.org, revised Jun 2024.
    12. Tomaz Cajner & Leland D. Crane & Ryan A. Decker & Adrian Hamins-Puertolas & Christopher Kurz, 2019. "Improving the Accuracy of Economic Measurement with Multiple Data Sources: The Case of Payroll Employment Data," NBER Chapters, in: Big Data for Twenty-First-Century Economic Statistics, pages 147-170, National Bureau of Economic Research, Inc.
    13. Parmar, Rashik & Leiponen, Aija & Thomas, Llewellyn D.W., 2020. "Building an organizational digital twin," Business Horizons, Elsevier, vol. 63(6), pages 725-736.
    14. Horvath, Akos & Kay, Benjamin & Wix, Carlo, 2023. "The COVID-19 shock and consumer credit: Evidence from credit card data," Journal of Banking & Finance, Elsevier, vol. 152(C).
    15. Chernozhukov, Victor & Kasahara, Hiroyuki & Schrimpf, Paul, 2021. "Causal impact of masks, policies, behavior on early covid-19 pandemic in the U.S," Journal of Econometrics, Elsevier, vol. 220(1), pages 23-62.
    16. W. Erwin Diewert & Kevin J. Fox, 2020. "Measuring Real Consumption and CPI Bias under Lockdown Conditions," NBER Working Papers 27144, National Bureau of Economic Research, Inc.
    17. Hall, Robert E. & Kudlyak, Marianna, 2022. "The unemployed with jobs and without jobs," Labour Economics, Elsevier, vol. 79(C).
    18. Autor, David & Cho, David & Crane, Leland D. & Goldar, Mita & Lutz, Byron & Montes, Joshua & Peterman, William B. & Ratner, David & Villar, Daniel & Yildirmaz, Ahu, 2022. "An evaluation of the Paycheck Protection Program using administrative payroll microdata," Journal of Public Economics, Elsevier, vol. 211(C).
    19. Nathaniel Lane, 2020. "The New Empirics of Industrial Policy," Journal of Industry, Competition and Trade, Springer, vol. 20(2), pages 209-234, June.
    20. Daniel Aaronson & Scott A. Brave & Michael Fogarty & Ezra Karger & Spencer D. Krane, 2021. "Tracking U.S. Consumers in Real Time with a New Weekly Index of Retail Trade," Working Paper Series WP-2021-05, Federal Reserve Bank of Chicago, revised 18 Jun 2021.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:bushor:v:65:y:2022:i:4:p:481-492. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/bushor .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.