IDEAS home Printed from https://ideas.repec.org/a/kap/compec/v35y2010i4p371-394.html
   My bibliography  Save this article

Should Economists Use Open Source Software for Doing Research?

Author

Abstract

We survey the literature on the accuracy of econometric software. We also assess the advantages of open source software from the point of view of reliability and discuss its potential in applied economics, which has now become fully dependent on computers. As a case study, we apply various accuracy tests on gretl (GNU Regression, Econometrics and Time-series Library) and demonstrate that the open source nature of the program made it possible to see the cause, facilitated a rapid fix, and enabled verifying the correction of a number of flaws that we uncovered. We also run the same tests on four widely-used proprietary econometric packages and observe the known accuracy errors that remained uncorrected for more than five years.
(This abstract was borrowed from another version of this item.)
(This abstract was borrowed from another version of this item.)

Suggested Citation

  • A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    • Handle: RePEc:kap:compec:v:35:y:2010:i:4:p:371-394
    DOI: 10.1007/s10614-010-9204-4
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s10614-010-9204-4
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1007/s10614-010-9204-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Choi, Hwan-sik & Kiefer, Nicholas M., 2005. "Software evaluation: EasyReg International," International Journal of Forecasting, Elsevier, vol. 21(3), pages 609-616.
    2. Josh Lerner, 2005. "The Scope of Open Source Licensing," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 21(1), pages 20-56, April.
    3. Silk, Julian, 1996. "Systems Estimation: A Comparison of SAS, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(4), pages 437-450, July-Aug..
    4. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    5. Christian Kleiber & Achim Zeileis, 2005. "Validating multiple structural change models-a case study," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(5), pages 685-690.
    6. Gita Persand & Chris Brooks & Simon P. Burke, 2003. "Multivariate GARCH models: software choice and estimation issues," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(6), pages 725-734.
    7. Josh Lerner & Jean Tirole, 2002. "Some Simple Economics of Open Source," Journal of Industrial Economics, Wiley Blackwell, vol. 50(2), pages 197-234, June.
    8. Ricardo De Bonis & Giuseppe Bruno, 2000. "A Comparative Study Of Alternative Econometric Packages: An Application To Italian Deposit Interest Rates," Computing in Economics and Finance 2000 160, Society for Computational Economics.
    9. Lerner, Josh & Tirole, Jean, 2001. "The open source movement: Key research questions," European Economic Review, Elsevier, vol. 45(4-6), pages 819-826, May.
    10. Josh Lerner & Jean Tirole, 2005. "The Economics of Technology Sharing: Open Source and Beyond," Journal of Economic Perspectives, American Economic Association, vol. 19(2), pages 99-120, Spring.
    11. B. D. McCullough & H. D. Vinod, 2003. "Econometrics and Software: Comments," Journal of Economic Perspectives, American Economic Association, vol. 17(1), pages 223-224, Winter.
    12. West, Joel, 2003. "How open is open enough?: Melding proprietary and open source platform strategies," Research Policy, Elsevier, vol. 32(7), pages 1259-1285, July.
    13. A. Talha Yalta & A. Yasemin Yalta, 2007. "GRETL 1.6.0 and its numerical accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(4), pages 849-854.
    14. J. Wilson Mixon Jr & Ryan J. Smith, 2006. "Teaching undergraduate econometrics with GRETL," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(7), pages 1103-1107, November.
    15. Sawitzki, Gunther, 1994. "Testing numerical reliability of data analysis systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 269-286, September.
    16. Giovanni Baiocchi & Walter Distaso, 2003. "GRETL: Econometric software for the GNU generation," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 105-110.
    17. Stefan Haefliger & Georg von Krogh & Sebastian Spaeth, 2008. "Code Reuse in Open Source Software," Management Science, INFORMS, vol. 54(1), pages 180-193, January.
    18. B. D. McCullough, 2009. "Testing Econometric Software," Palgrave Macmillan Books, in: Terence C. Mills & Kerry Patterson (ed.), Palgrave Handbook of Econometrics, chapter 28, pages 1293-1320, Palgrave Macmillan.
    19. B. A. Wichmann & I. D. Hill, 1982. "An Efficient and Portable Pseudo‐Random Number Generator," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 31(2), pages 188-190, June.
    20. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.
    21. Newbold, Paul & Agiakloglou, Christos & Miller, John, 1994. "Adventures with ARIMA software," International Journal of Forecasting, Elsevier, vol. 10(4), pages 573-581, December.
    22. Hertel, Guido & Niedner, Sven & Herrmann, Stefanie, 2003. "Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel," Research Policy, Elsevier, vol. 32(7), pages 1159-1177, July.
    23. Brooks, Chris & Burke, Simon P. & Persand, Gita, 2001. "Benchmarks and the accuracy of GARCH model estimation," International Journal of Forecasting, Elsevier, vol. 17(1), pages 45-56.
    24. von Krogh, Georg & Spaeth, Sebastian & Lakhani, Karim R., 2003. "Community, joining, and specialization in open source software innovation: a case study," Research Policy, Elsevier, vol. 32(7), pages 1217-1241, July.
    25. Altman, Micah & McDonald, Michael P., 2003. "Replication with Attention to Numerical Accuracy," Political Analysis, Cambridge University Press, vol. 11(3), pages 302-307, July.
    26. A. Talha Yalta, 2010. "The Accuracy of Statistical Distributions in Microsoft (R) Excel 2007," Working Papers 1006, TOBB University of Economics and Technology, Department of Economics.
    27. McCullough, B D, 1999. "Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(2), pages 191-202, March-Apr.
    28. Yalta, A. Talha & Jenal, Olaf, 2009. "On the importance of verifying forecasting results," International Journal of Forecasting, Elsevier, vol. 25(1), pages 62-73.
    29. Josh Lerner & Parag A. Pathak & Jean Tirole, 2006. "The Dynamics of Open-Source Contributors," American Economic Review, American Economic Association, vol. 96(2), pages 114-118, May.
    30. Knusel, Leo, 1995. "On the accuracy of the statistical distributions in GAUSS," Computational Statistics & Data Analysis, Elsevier, vol. 20(6), pages 699-702, December.
    31. H. D. Vinod, 2000. "Review of GAUSS for Windows, including its numerical accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(2), pages 211-220.
    32. A. Talha Yalta & Riccardo Lucchetti, 2008. "The GNU|Linux platform and freedom respecting software for economists," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 23(2), pages 279-286.
    33. Richard Anderson & William Greene & B. D. McCullough & H. D. Vinod, 2008. "The role of data/code archives in the future of economic research," Journal of Economic Methodology, Taylor & Francis Journals, vol. 15(1), pages 99-119.
    34. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    35. D. McCullough, B. & Wilson, Berry, 2002. "On the accuracy of statistical procedures in Microsoft Excel 2000 and Excel XP," Computational Statistics & Data Analysis, Elsevier, vol. 40(4), pages 713-721, October.
    36. Knusel, Leo, 2005. "On the accuracy of statistical distributions in Microsoft Excel 2003," Computational Statistics & Data Analysis, Elsevier, vol. 48(3), pages 445-449, March.
    37. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    38. McCullough, B.D., 2008. "Microsoft Excel's 'Not The Wichmann-Hill' random number generators," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4587-4593, June.
    39. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    40. B. D. McCullough, 1999. "Wilkinson's Tests and Econometric Software," Computing in Economics and Finance 1999 1312, Society for Computational Economics.
    41. McCullough, B. D. & Wilson, Berry, 1999. "On the accuracy of statistical procedures in Microsoft Excel 97," Computational Statistics & Data Analysis, Elsevier, vol. 31(1), pages 27-37, July.
    42. Sawitzki, Gunther, 1994. "Report on the Numerical Reliability of Data Analysis Systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 289-301, September.
    43. McCullough, B.D. & Wilson, Berry, 2005. "On the accuracy of statistical procedures in Microsoft Excel 2003," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1244-1252, June.
    44. Knusel, Leo, 1998. "On the accuracy of statistical distributions in Microsoft Excel 97," Computational Statistics & Data Analysis, Elsevier, vol. 26(3), pages 375-377, January.
    45. H. D. Vinod & B. D. McCullough, 1999. "The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(2), pages 633-665, June.
    46. A. M. Kitchen & R. Drachenberg & J. Symanzik, 2003. "Assessing the reliability of web-based statistical software," Computational Statistics, Springer, vol. 18(1), pages 107-122, March.
    47. Vinod, H. D., 2001. "Care and feeding of reproducible econometrics," Journal of Econometrics, Elsevier, vol. 100(1), pages 87-88, January.
    48. H. D. Vinod & B. D. McCullough, 1999. "Corrigenda: The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1565-1565, December.
    49. Riccardo Lucchetti, 2009. "Who uses gretl? An Analysis of the SourceForge Download Data," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 3, pages 45-55, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    50. Nerlove, Marc, 2003. "Programming Languages: A Short History For Economists," Working Papers 28555, University of Maryland, Department of Agricultural and Resource Economics.
    51. Jeffrey A. Roberts & Il-Horn Hann & Sandra A. Slaughter, 2006. "Understanding the Motivations, Participation, and Performance of Open Source Software Developers: A Longitudinal Study of the Apache Projects," Management Science, INFORMS, vol. 52(7), pages 984-999, July.
    52. Johnson, Justin P., 2006. "Collaboration, peer review and open source software," Information Economics and Policy, Elsevier, vol. 18(4), pages 477-497, November.
    53. Lakhani, Karim R. & von Hippel, Eric, 2003. "How open source software works: "free" user-to-user assistance," Research Policy, Elsevier, vol. 32(6), pages 923-943, June.
    54. Andrea Bonaccorsi & Silvia Giannangeli & Cristina Rossi, 2006. "Entry Strategies Under Competing Standards: Hybrid Business Models in the Open Source Software Industry," Management Science, INFORMS, vol. 52(7), pages 1085-1098, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Blog mentions

    As found by EconAcademics.org, the blog aggregator for Economics research:
    1. On the advantages of open-source econometrics
      by Economic Logician in Economic Logic on 2012-02-13 21:08:00

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    2. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    3. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01, June.
    4. H.-J. Sun & Kaoru Fukuda & B. D. McCullough, 2017. "Inaccurate regression coefficients in Microsoft Excel 2003: an investigation of Volpi’s “zero bug”," Computational Statistics, Springer, vol. 32(4), pages 1411-1421, December.
    5. Rodolphe Buda, 2013. "SIMUL 3.2: An Econometric Tool for Multidimensional Modelling," Computational Economics, Springer;Society for Computational Economics, vol. 41(4), pages 517-524, April.
    6. Ho, Anson T.Y. & Huynh, Kim P. & Jacho-Chávez, David T., 2019. "Using nonparametric copulas to measure crude oil price co-movements," Energy Economics, Elsevier, vol. 82(C), pages 211-223.
    7. Yalta, A. Talha & Schreiber, Sven, 2012. "Random Number Generation in gretl," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(c01).
    8. repec:asg:wpaper:1047 is not listed on IDEAS
    9. Luc Anselin, 2012. "From SpaceStat to CyberGIS," International Regional Science Review, , vol. 35(2), pages 131-157, April.
    10. el Alaoui, AbdelKader Ouatik & Bacha, Obiyathulla Ismath & Masih, Mansur & Asutay, Mehmet, 2016. "Shari’ah screening, market risk and contagion: A multi-country analysis," Journal of Economic Behavior & Organization, Elsevier, vol. 132(S), pages 93-112.
    11. Rodolphe Buda, 2015. "Data Checking and Econometric Software Development: A Technique of Traceability by Fictive Data Encoding," Computational Economics, Springer;Society for Computational Economics, vol. 46(2), pages 325-357, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yalta, A. Talha & Jenal, Olaf, 2009. "On the importance of verifying forecasting results," International Journal of Forecasting, Elsevier, vol. 25(1), pages 62-73.
    2. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    3. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01, June.
    4. Charles G. Renfro, 2009. "The Practice of Econometric Theory," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75571-5.
    5. Oluwarotimi O. Odeh & Allen M. Featherstone & Jason S. Bergtold, 2010. "Reliability of Statistical Software," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(5), pages 1472-1489.
    6. McCullough, Bruce D. & Yalta, A. Talha, 2013. "Spreadsheets in the Cloud - Not Ready Yet," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 52(i07).
    7. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    8. Engelhardt, Sebastian v. & Freytag, Andreas, 2013. "Institutions, culture, and open source," Journal of Economic Behavior & Organization, Elsevier, vol. 95(C), pages 90-110.
    9. repec:jss:jstsof:34:i04 is not listed on IDEAS
    10. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    11. Maha Shaikh & Emmanuelle Vaast, 2016. "Folding and Unfolding: Balancing Openness and Transparency in Open Source Communities," Information Systems Research, INFORMS, vol. 27(4), pages 813-833, December.
    12. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    13. David M. Waguespack & Lee Fleming, 2009. "Scanning the Commons? Evidence on the Benefits to Startups Participating in Open Standards Development," Management Science, INFORMS, vol. 55(2), pages 210-223, February.
    14. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    15. Yalta, A. Talha & Schreiber, Sven, 2012. "Random Number Generation in gretl," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(c01).
    16. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    17. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    18. Hargreaves, Bruce R. & McWilliams, Thomas P., 2010. "Polynomial Trendline function flaws in Microsoft Excel," Computational Statistics & Data Analysis, Elsevier, vol. 54(4), pages 1190-1196, April.
    19. Islam, Mazhar & Miller, Jacob & Park, Haemin Dennis, 2017. "But what will it cost me? How do private costs of participation affect open source software projects?," Research Policy, Elsevier, vol. 46(6), pages 1062-1070.
    20. Fabio M. Manenti & Stefano Comino & Marialaura Parisi, 2005. "From Planning to Mature: on the Determinants of Open Source Take-Off," Industrial Organization 0507006, University Library of Munich, Germany, revised 29 Sep 2005.
    21. A. Talha Yalta & A. Yasemin Yalta, 2007. "GRETL 1.6.0 and its numerical accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(4), pages 849-854.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:kap:compec:v:35:y:2010:i:4:p:371-394. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.