IDEAS home Printed from https://ideas.repec.org/a/oup/ajagec/v92y2010i5p1472-1489.html
   My bibliography  Save this article

Reliability of Statistical Software

Author

Listed:
  • Oluwarotimi O. Odeh
  • Allen M. Featherstone
  • Jason S. Bergtold

Abstract

The reliability of several statisitcal software packages was examined using the National Institute of Standards and Technology linear and nonlinear least squares datasets and models. Software tested include Excel 2007, GAMS 23.4, GAUSS 9.0, LIMDEP 8.0, Mathematica 7.0, MATLAB 7.5, R 2.10, SAS 9.1, SHAZAM 10, and Stata 10. While some of these packages have been previously examined, others, including GAMS and MATLAB, have not been extensively examined. Reliability tests indicate improvements in some of the software packages that were previously tested, but some of these packages failed reliability tests under certain conditions. The findings underscore the need to benchmark software packages to ascertain reliability before use and the importance of solving econometric problems using more than one package. Copyright 2010, Oxford University Press.

Suggested Citation

  • Oluwarotimi O. Odeh & Allen M. Featherstone & Jason S. Bergtold, 2010. "Reliability of Statistical Software," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 92(5), pages 1472-1489.
    • Handle: RePEc:oup:ajagec:v:92:y:2010:i:5:p:1472-1489
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/ajae/aaq068
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Silk, Julian, 1996. "Systems Estimation: A Comparison of SAS, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 11(4), pages 437-450, July-Aug..
    2. Yalta, A. Talha, 2007. "The Numerical Reliability of GAUSS 8.0," The American Statistician, American Statistical Association, vol. 61, pages 262-268, August.
    3. A. Talha Yalta, 2010. "The Accuracy of Statistical Distributions in Microsoft (R) Excel 2007," Working Papers 1006, TOBB University of Economics and Technology, Department of Economics.
    4. McCullough, B D, 1999. "Econometric Software Reliability: EViews, LIMDEP, SHAZAM and TSP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 14(2), pages 191-202, March-Apr.
    5. H. D. Vinod, 2000. "Review of GAUSS for Windows, including its numerical accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 15(2), pages 211-220.
    6. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    7. Davidson, Russell & MacKinnon, James G., 1993. "Estimation and Inference in Econometrics," OUP Catalogue, Oxford University Press, number 9780195060119.
    8. B. D. McCullough, 2006. "A review of TESTU01," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 21(5), pages 677-682, July.
    9. Sawitzki, Gunther, 1994. "Testing numerical reliability of data analysis systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 269-286, September.
    10. Thomas F. Tice & Marilyn G. Kletke, 1984. "Reliability of Linear Programming Software: An Experience with the IBM Mathematical Programming System Series," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 66(1), pages 104-107.
    11. McCullough, B.D. & Heiser, David A., 2008. "On the accuracy of statistical procedures in Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4570-4578, June.
    12. Sawitzki, Gunther, 1994. "Report on the Numerical Reliability of Data Analysis Systems," Computational Statistics & Data Analysis, Elsevier, vol. 18(2), pages 289-301, September.
    13. McCullough, B.D. & Wilson, Berry, 2005. "On the accuracy of statistical procedures in Microsoft Excel 2003," Computational Statistics & Data Analysis, Elsevier, vol. 49(4), pages 1244-1252, June.
    14. William G. Tomek, 1993. "Confirmation and Replication in Empirical Econometrics: A Step Toward Improved Scholarship," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 75(Special_I), pages 6-14.
    15. H. D. Vinod & B. D. McCullough, 1999. "The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(2), pages 633-665, June.
    16. H. D. Vinod & B. D. McCullough, 1999. "Corrigenda: The Numerical Reliability of Econometric Software," Journal of Economic Literature, American Economic Association, vol. 37(4), pages 1565-1565, December.
    17. Simon, Stephen D. & Lesage, James P., 1988. "Benchmarking numerical accuracy of statistical algorithms," Computational Statistics & Data Analysis, Elsevier, vol. 7(2), pages 197-209, December.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. D. Betsy McCoach & Graham G. Rifenbark & Sarah D. Newton & Xiaoran Li & Janice Kooken & Dani Yomtov & Anthony J. Gambino & Aarti Bellara, 2018. "Does the Package Matter? A Comparison of Five Common Multilevel Modeling Software Packages," Journal of Educational and Behavioral Statistics, , vol. 43(5), pages 594-627, October.
    2. Jason S. Bergtold & Krishna P. Pokharel & Allen M. Featherstone & Lijia Mo, 2018. "On the examination of the reliability of statistical software for estimating regression models with discrete dependent variables," Computational Statistics, Springer, vol. 33(2), pages 757-786, June.
    3. Bergtold, Jason S. & Pokharel, Krishna & Featherstone, Allen, 2015. "On the Examination of the Reliability of Statistical Software for Estimating Logistic Regression Models," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205643, Agricultural and Applied Economics Association.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. A. Yalta & A. Yalta, 2010. "Should Economists Use Open Source Software for Doing Research?," Computational Economics, Springer;Society for Computational Economics, vol. 35(4), pages 371-394, April.
    2. Yalta, A. Talha & Jenal, Olaf, 2009. "On the importance of verifying forecasting results," International Journal of Forecasting, Elsevier, vol. 25(1), pages 62-73.
    3. Charles G. Renfro, 2009. "The Practice of Econometric Theory," Advanced Studies in Theoretical and Applied Econometrics, Springer, number 978-3-540-75571-5.
    4. A. M. Kitchen & R. Drachenberg & J. Symanzik, 2003. "Assessing the reliability of web-based statistical software," Computational Statistics, Springer, vol. 18(1), pages 107-122, March.
    5. Bergtold, Jason S. & Pokharel, Krishna & Featherstone, Allen, 2015. "On the Examination of the Reliability of Statistical Software for Estimating Logistic Regression Models," 2015 AAEA & WAEA Joint Annual Meeting, July 26-28, San Francisco, California 205643, Agricultural and Applied Economics Association.
    6. McCullough, B. D., 2000. "Is it safe to assume that software is accurate?," International Journal of Forecasting, Elsevier, vol. 16(3), pages 349-357.
    7. McCullough, B.D., 2008. "Special section on Microsoft Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4568-4569, June.
    8. Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), 2009. "Econometrics with gretl. Proceedings of the gretl Conference 2009," UPV/EHU Books, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales, edition 1, number 01, June.
    9. A. Talha Yalta & A. Yasemin Yalta, 2009. "Wilkinson Tests and gretl," EHUCHAPS, in: Ignacio Díaz-Emparanza & Petr Mariel & María Victoria Esteban (ed.), Econometrics with gretl. Proceedings of the gretl Conference 2009, edition 1, chapter 16, pages 243-251, Universidad del País Vasco - Facultad de Ciencias Económicas y Empresariales.
    10. Guy Mélard, 2014. "On the accuracy of statistical procedures in Microsoft Excel 2010," Computational Statistics, Springer, vol. 29(5), pages 1095-1128, October.
    11. Jason S. Bergtold & Krishna P. Pokharel & Allen M. Featherstone & Lijia Mo, 2018. "On the examination of the reliability of statistical software for estimating regression models with discrete dependent variables," Computational Statistics, Springer, vol. 33(2), pages 757-786, June.
    12. Ooms, M., 2008. "Trends in Applied Econometrics Software Development 1985-2008, an analysis of Journal of Applied Econometrics research articles, software reviews, data and code," Serie Research Memoranda 0021, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    13. Gilles Teyssière, 2005. "Structural time series modelling with STAMP 6.02," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(4), pages 571-577, May.
    14. Yalta, A. Talha, 2008. "The accuracy of statistical distributions in Microsoft® Excel 2007," Computational Statistics & Data Analysis, Elsevier, vol. 52(10), pages 4579-4586, June.
    15. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    16. Yalta, A. Talha & Schreiber, Sven, 2012. "Random Number Generation in gretl," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 50(c01).
    17. F. Javier Mencía & Enrique Sentana, 2004. "Estimation and Testing of Dynamic Models with Generalised Hyperbolic Innovations," Working Papers wp2004_0411, CEMFI.
    18. Mencía, Javier & Sentana, Enrique, 2009. "Multivariate location-scale mixtures of normals and mean-variance-skewness portfolio allocation," Journal of Econometrics, Elsevier, vol. 153(2), pages 105-121, December.
    19. A. Talha Yalta & A. Yasemin Yalta, 2007. "GRETL 1.6.0 and its numerical accuracy," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 22(4), pages 849-854.
    20. Keeling, Kellie B. & Pavur, Robert J., 2007. "A comparative study of the reliability of nine statistical software packages," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3811-3831, May.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:ajagec:v:92:y:2010:i:5:p:1472-1489. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://edirc.repec.org/data/aaeaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.