IDEAS home Printed from https://ideas.repec.org/a/aiy/jnjaer/v20y2021i2p294-326.html
   My bibliography  Save this article

Review of Modern Approaches for Assessing the Effectiveness of Banking

Author

Listed:
  • M.V. Leonov

Abstract

In the current circumstances of digital transformation of the economy, the study of the efficiency of commercial banks is widely demanded and provides an opportunity for identifying the prerequisites for the formation of a sustainable financial system. Despite the large number of scientific publications in this field, there are no systematic studies that summarize the existing methodological framework for the study of the effectiveness of banking activities. The purpose of the article is to critically analyze the approaches to assessing the efficiency of banking activities and prospects for its improvement in the digital economy. The complexity of assessing the effectiveness of commercial banks consists in the multiplicity of forms of output and resources used for this process. In this study, the author puts forward a hypothesis that the implementation of modern approaches and methods to assess the effectiveness of banking activities can improve the accuracy of the assessment itself, and help identify factors that increase the effectiveness of such activities. The article is based on English-language scientific papers published in between 2016 and 2020 and indexed in the international scientometric database «Web of Science». The author applies the method of systematic bibliographic study of the set of publications, highlights the main discussion issues of the recent years, namely, research methodology, internal efficiency factors, country specifics of banking activities. The article reveals the content, certain issues, and the limits of the applicability of the nonparametric method of data envelopment analysis and the parametric method of stochastic frontier analysis in assessing the effectiveness of activities. Particular attention is paid to the generalization of research results in terms of identifying factors that have a significant impact on the effectiveness of commercial banks: ownership structure, returns to scale, regulation. The author highlights the increasing role of information technology as a key production factor, highlights the phenomenon of breaking the value chain in banking activities, the formation of new business models and the functioning of banking ecosystems. The scientific and practical significance of the article lies in the gain of knowledge, which might provide the basis for the development of measures to improve banking regulation, as well as serve as the framework for the identification of the most effective forms of banking intermediation.

Suggested Citation

  • M.V. Leonov, 2021. "Review of Modern Approaches for Assessing the Effectiveness of Banking," Journal of Applied Economic Research, Graduate School of Economics and Management, Ural Federal University, vol. 20(2), pages 294-326.
  • Handle: RePEc:aiy:jnjaer:v:20:y:2021:i:2:p:294-326
    DOI: http://dx.doi.org/10.15826/vestnik.2021.20.2.013
    as

    Download full text from publisher

    File URL: https://journalaer.ru//fileadmin/user_upload/site_15934/2021/2_20_2021_Leonov.pdf
    Download Restriction: no

    File URL: https://libkey.io/http://dx.doi.org/10.15826/vestnik.2021.20.2.013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Sarmiento, Miguel & Galán, Jorge E., 2017. "The influence of risk-taking on bank efficiency: Evidence from Colombia," Emerging Markets Review, Elsevier, vol. 32(C), pages 52-73.
    2. Zha, Yong & Liang, Nannan & Wu, Maoguo & Bian, Yiwen, 2016. "Efficiency evaluation of banks in China: A dynamic two-stage slacks-based measure approach," Omega, Elsevier, vol. 60(C), pages 60-72.
    3. Berger, Allen N. & Humphrey, David B., 1997. "Efficiency of financial institutions: International survey and directions for future research," European Journal of Operational Research, Elsevier, vol. 98(2), pages 175-212, April.
    4. repec:eme:ijlma0:ijlma-01-2016-0008 is not listed on IDEAS
    5. María Concepción Pérez-Cárceles & Juan Cándido Gómez-Gallego & Juan Gómez-García, 2016. "Distribution of cost inefficiency in stochastic frontier approach: evidence from Spanish banking," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(16), pages 3030-3041, December.
    6. Abdel Latef M. Anouze & Imad Bou-Hamad, 2019. "Data envelopment analysis and data mining to efficiency estimation and evaluation," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(2), pages 169-190, April.
    7. Alois Kneip & Léopold Simar & Paul W. Wilson, 2016. "Testing Hypotheses in Nonparametric Models of Production," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 34(3), pages 435-456, July.
    8. Hakan Gunes & Dilem Yildirim, 2016. "Estimating cost efficiency of Turkish commercial banks under unobserved heterogeneity with stochastic frontier models," Central Bank Review, Research and Monetary Policy Department, Central Bank of the Republic of Turkey, vol. 16(4), pages 127-136.
    9. repec:eme:mfppss:v:42:y:2016:i:10:p:980-998 is not listed on IDEAS
    10. repec:eme:ijlma0:ijlma-11-2016-0142 is not listed on IDEAS
    11. Abdel Latef M. Anouze & Imad Bou-Hamad, 2019. "Data envelopment analysis and data mining to efficiency estimation and evaluation," International Journal of Islamic and Middle Eastern Finance and Management, Emerald Group Publishing Limited, vol. 12(2), pages 169-190, April.
    12. Atwood, Joseph & Shaik, Saleem, 2020. "Theory and statistical properties of Quantile Data Envelopment Analysis," European Journal of Operational Research, Elsevier, vol. 286(2), pages 649-661.
    13. Zsuzsanna Hosszú & Bálint Dancsik, 2018. "Measuring Bank Efficiency and Market Power in the Household and Corporate Credit Markets Considering Credit Risks," Acta Oeconomica, Akadémiai Kiadó, Hungary, vol. 68(2), pages 175-207, June.
    14. Jayakumar, Manju & Pradhan, Rudra P. & Dash, Saurav & Maradana, Rana P. & Gaurav, Kunal, 2018. "Banking competition, banking stability, and economic growth: Are feedback effects at work?," Journal of Economics and Business, Elsevier, vol. 96(C), pages 15-41.
    15. Asmild, Mette & Zhu, Minyan, 2016. "Controlling for the use of extreme weights in bank efficiency assessments during the financial crisis," European Journal of Operational Research, Elsevier, vol. 251(3), pages 999-1015.
    16. Suneeta Sathye & Milind Sathye, 2017. "Do ATMs Increase Technical Efficiency of Banks in a Developing Country? Evidence from Indian Banks," Australian Accounting Review, CPA Australia, vol. 27(1), pages 101-111, March.
    17. Thanh Ngo & David Tripe, 2017. "Measuring efficiency of Vietnamese banks," Pacific Accounting Review, Emerald Group Publishing Limited, vol. 29(2), pages 171-182, April.
    18. Adnan Kasman & Kamila Mekenbayeva, 2016. "Technical Efficiency and Total Factor Productivity in the Kazakh Banking Industry," Acta Oeconomica, Akadémiai Kiadó, Hungary, vol. 66(4), pages 685-709, December.
    19. Timothy J. Coelli & D.S. Prasada Rao & Christopher J. O’Donnell & George E. Battese, 2005. "An Introduction to Efficiency and Productivity Analysis," Springer Books, Springer, edition 0, number 978-0-387-25895-9, July.
    20. Huang, Tai-Hsin & Hu, Chu-Nan & Chang, Bao-Guang, 2018. "Competition, efficiency, and innovation in Taiwan’s banking industry — An application of copula methods," The Quarterly Review of Economics and Finance, Elsevier, vol. 67(C), pages 362-375.
    21. Osuagwu, Eze Simpson & Isola, Wakeel & Nwaogwugwu, Isaac, 2018. "Measuring Technical Efficiency and Productivity Change in the Nigerian Banking Sector: A Comparison of non-parametric DEA and parametric SFA," MPRA Paper 112948, University Library of Munich, Germany.
    22. Chu, Junfei & Wu, Jie & Chu, Chengbin & Zhang, Tinglong, 2020. "DEA-based fixed cost allocation in two-stage systems: Leader-follower and satisfaction degree bargaining game approaches," Omega, Elsevier, vol. 94(C).
    23. Vaneet Bhatia & Sankarshan Basu & Subrata Kumar Mitra & Pradyumna Dash, 2018. "A review of bank efficiency and productivity," OPSEARCH, Springer;Operational Research Society of India, vol. 55(3), pages 557-600, November.
    24. Mamonov, Mikhail & Vernikov, Andrei, 2017. "Bank ownership and cost efficiency: New empirical evidence from Russia," Economic Systems, Elsevier, vol. 41(2), pages 305-319.
    25. Tsionas, Mike G. & Assaf, A. George & Andrikopoulos, Athanasios, 2020. "Quantile stochastic frontier models with endogeneity," Economics Letters, Elsevier, vol. 188(C).
    26. Natalya Zelenyuk & Valentin Zelenyuk, 2021. "Bank Performance Analysis," CEPA Working Papers Series WP022021, School of Economics, University of Queensland, Australia.
    27. Doan, Anh-Tuan & Lin, Kun-Li & Doong, Shuh-Chyi, 2018. "What drives bank efficiency? The interaction of bank income diversification and ownership," International Review of Economics & Finance, Elsevier, vol. 55(C), pages 203-219.
    28. Degl'Innocenti, Marta & Kourtzidis, Stavros A. & Sevic, Zeljko & Tzeremes, Nickolaos G., 2017. "Investigating bank efficiency in transition economies: A window-based weight assurance region approach," Economic Modelling, Elsevier, vol. 67(C), pages 23-33.
    29. Kekoura Sakouvogui, 2020. "A comparative approach of stochastic frontier analysis and data envelopment analysis estimators: evidence from banking system," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 47(7), pages 1787-1810, May.
    30. Khan, Habib Hussain & Kutan, Ali M. & Naz, Iram & Qureshi, Fiza, 2017. "Efficiency, growth and market power in the banking industry: New approach to efficient structure hypothesis," The North American Journal of Economics and Finance, Elsevier, vol. 42(C), pages 531-545.
    31. Fukuyama, Hirofumi & Matousek, Roman & Tzeremes, Nickolaos G., 2020. "A Nerlovian cost inefficiency two-stage DEA model for modeling banks’ production process: Evidence from the Turkish banking system," Omega, Elsevier, vol. 95(C).
    32. Fethi, Meryem Duygun & Pasiouras, Fotios, 2010. "Assessing bank efficiency and performance with operational research and artificial intelligence techniques: A survey," European Journal of Operational Research, Elsevier, vol. 204(2), pages 189-198, July.
    33. Md. Abul Kalam Azad & Peter Wanke & Mohammad Zahir Raihan & S.M. Rakibul Anwar & Riduanul Mustafa, 2020. "Bank efficiency in Bangladesh revisited: a slack-based network DEA approach," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 47(5), pages 1001-1014, April.
    34. Tsionas, Mike G., 2020. "Quantile Stochastic Frontiers," European Journal of Operational Research, Elsevier, vol. 282(3), pages 1177-1184.
    35. Glass, Anthony J. & Kenjegalieva, Karligash, 2019. "A spatial productivity index in the presence of efficiency spillovers: Evidence for U.S. banks, 1992–2015," European Journal of Operational Research, Elsevier, vol. 273(3), pages 1165-1179.
    36. Chen, Ya & Li, Yongjun & Liang, Liang & Salo, Ahti & Wu, Huaqing, 2016. "Frontier projection and efficiency decomposition in two-stage processes with slacks-based measures," European Journal of Operational Research, Elsevier, vol. 250(2), pages 543-554.
    37. Lee, Chi-Chuan & Huang, Tai-Hsin, 2017. "Cost efficiency and technological gap in Western European banks: A stochastic metafrontier analysis," International Review of Economics & Finance, Elsevier, vol. 48(C), pages 161-178.
    38. Tavakoli, Ibrahim M. & Mostafaee, Amin, 2019. "Free disposal hull efficiency scores of units with network structures," European Journal of Operational Research, Elsevier, vol. 277(3), pages 1027-1036.
    39. Hakan Güneş & Dilem Yıldırım, 2016. "Estimating Cost Efficiency of Turkish Commercial Banks under Unobserved Heterogeneity with Stochastic Frontier Models," ERC Working Papers 1603, ERC - Economic Research Center, Middle East Technical University, revised Mar 2016.
    40. Eze Simpson Osuagwu & Wakeel Atanda Isola & Isaac Chii Nwaogwugwu, 2018. "Measuring Technical Efficiency and Productivity Change in the Nigerian Banking Sector: A Comparison of Non‐parametric and Parametric Techniques," African Development Review, African Development Bank, vol. 30(4), pages 490-501, December.
    41. Wanke, Peter & Abul Kalam Azad, Md & Emrouznejad, Ali & Antunes, Jorge, 2019. "A dynamic network DEA model for accounting and financial indicators: A case of efficiency in MENA banking," International Review of Economics & Finance, Elsevier, vol. 61(C), pages 52-68.
    42. Abreu, Emmanuel Sousa de & Kimura, Herbert & Sobreiro, Vinicius Amorim, 2019. "What is going on with studies on banking efficiency?," Research in International Business and Finance, Elsevier, vol. 47(C), pages 195-219.
    43. Wanke, Peter & Azad, Abul Kalam & Emrouznejad, Ali, 2018. "Efficiency in BRICS banking under data vagueness: A two-stage fuzzy approach," Global Finance Journal, Elsevier, vol. 35(C), pages 58-71.
    44. Zhou, Xiaoyang & Xu, Zhongwen & Chai, Jian & Yao, Liming & Wang, Shouyang & Lev, Benjamin, 2019. "Efficiency evaluation for banking systems under uncertainty: A multi-period three-stage DEA model," Omega, Elsevier, vol. 85(C), pages 68-82.
    45. Quaranta, Anna Grazia & Raffoni, Anna & Visani, Franco, 2018. "A multidimensional approach to measuring bank branch efficiency," European Journal of Operational Research, Elsevier, vol. 266(2), pages 746-760.
    46. Soheila Seyedboveir & Sohrab Kordrostami & Behrouz Daneshian & Alireza Amirteimoori, 2017. "Cost Efficiency Measurement in Data Envelopment Analysis with Dynamic Network Structures: A Relational Model," Asia-Pacific Journal of Operational Research (APJOR), World Scientific Publishing Co. Pte. Ltd., vol. 34(05), pages 1-13, October.
    47. repec:eme:par000:par-06-2016-0064 is not listed on IDEAS
    48. Wanke, Peter & Azad, Md. Abul Kalam & Barros, Carlos Pestana & Hassan, M. Kabir, 2016. "Predicting efficiency in Islamic banks: An integrated multicriteria decision making (MCDM) approach," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 45(C), pages 126-141.
    49. Fries, Steven & Taci, Anita, 2005. "Cost efficiency of banks in transition: Evidence from 289 banks in 15 post-communist countries," Journal of Banking & Finance, Elsevier, vol. 29(1), pages 55-81, January.
    50. Al-Gasaymeh, Anwar, 2016. "Bank efficiency determinant: Evidence from the gulf cooperation council countries," Research in International Business and Finance, Elsevier, vol. 38(C), pages 214-223.
    51. Felisitas Defung & Ruhul Salim & Harry Bloch, 2016. "Has regulatory reform had any impact on bank efficiency in Indonesia? A two-stage analysis," Applied Economics, Taylor & Francis Journals, vol. 48(52), pages 5060-5074, November.
    52. Jradi, Samah & Ruggiero, John, 2019. "Stochastic data envelopment analysis: A quantile regression approach to estimate the production frontier," European Journal of Operational Research, Elsevier, vol. 278(2), pages 385-393.
    53. Kutlu, Levent & Tran, Kien C. & Tsionas, Mike G., 2020. "A spatial stochastic frontier model with endogenous frontier and environmental variables," European Journal of Operational Research, Elsevier, vol. 286(1), pages 389-399.
    54. Lina Novickytė & Jolanta Droždz, 2018. "Measuring the Efficiency in the Lithuanian Banking Sector: The DEA Application," IJFS, MDPI, vol. 6(2), pages 1-15, March.
    55. Carlos Pestana Barros & Emanuel Reis Leão & Nkanga Pedro João Macanda & Zorro Mendes, 2016. "A Bayesian Efficiency Analysis of Angolan Banks," South African Journal of Economics, Economic Society of South Africa, vol. 84(3), pages 484-498, September.
    56. Yang Li & Yi-Kai Chen & Feng Sheng Chien & Wen Chih Lee & Yi Ching Hsu, 2016. "Study of optimal capital adequacy ratios," Journal of Productivity Analysis, Springer, vol. 45(3), pages 261-274, June.
    57. Necmi K. Avkiran & Yushu Zhu & David W. L. Tripe & Kathleen Walsh, 2017. "Can foreign banks compete in China?," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 57(4), pages 961-980, December.
    58. Zhenni Yang & Christopher Gan & Zhaohua Li, 2019. "Role of Bank Regulation on Bank Performance: Evidence from Asia-Pacific Commercial Banks," JRFM, MDPI, vol. 12(3), pages 1-25, August.
    59. Russell Kashian & Richard McGregory & Robert Drago, 2017. "Minority owned banks and efficiency revisited," Journal of Productivity Analysis, Springer, vol. 48(2), pages 97-116, December.
    60. Phung, Manh-Trung & Cheng, Cheng-Ping & Guo, Chuanyin & Kao, Chen-Yu, 2020. "Mixed Network DEA with Shared Resources: A Case of Measuring Performance for Banking Industry," Operations Research Perspectives, Elsevier, vol. 7(C).
    61. Asish Saha & Umakant Dash, 2016. "Consolidation in Indian banking: Does size matter?," DECISION: Official Journal of the Indian Institute of Management Calcutta, Springer;Indian Institute of Management Calcutta, vol. 43(3), pages 223-238, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Doumpos, Michalis & Zopounidis, Constantin & Gounopoulos, Dimitrios & Platanakis, Emmanouil & Zhang, Wenke, 2023. "Operational research and artificial intelligence methods in banking," European Journal of Operational Research, Elsevier, vol. 306(1), pages 1-16.
    2. Vaneet Bhatia & Sankarshan Basu & Subrata Kumar Mitra & Pradyumna Dash, 2018. "A review of bank efficiency and productivity," OPSEARCH, Springer;Operational Research Society of India, vol. 55(3), pages 557-600, November.
    3. Partovi, Elmira & Matousek, Roman, 2019. "Bank efficiency and non-performing loans: Evidence from Turkey," Research in International Business and Finance, Elsevier, vol. 48(C), pages 287-309.
    4. Kai Sun & Ruhul Salim, 2020. "A semiparametric stochastic input distance frontier model with application to the Indonesian banking industry," Journal of Productivity Analysis, Springer, vol. 54(2), pages 139-156, December.
    5. E. Fusco & R. Benedetti & F. Vidoli, 2023. "Stochastic frontier estimation through parametric modelling of quantile regression coefficients," Empirical Economics, Springer, vol. 64(2), pages 869-896, February.
    6. Maya El Hourani, 2022. "La capitalisation bancaire et la qualité institutionnelle détermine-t-elle l’efficience coût des banques de la région MENA ?," GREDEG Working Papers 2022-41, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    7. Alves, André Bernardo & Wanke, Peter & Antunes, Jorge & Chen, Zhongfei, 2020. "Endogenous network efficiency, macroeconomy, and competition: Evidence from the Portuguese banking industry," The North American Journal of Economics and Finance, Elsevier, vol. 52(C).
    8. Camanho, Ana Santos & Silva, Maria Conceicao & Piran, Fabio Sartori & Lacerda, Daniel Pacheco, 2024. "A literature review of economic efficiency assessments using Data Envelopment Analysis," European Journal of Operational Research, Elsevier, vol. 315(1), pages 1-18.
    9. Dan Li & Yanfeng Li & Yeming Gong & Jiawei Yang, 2021. "Estimation of bank performance from multiple perspectives: an alternative solution to the deposit dilemma," Journal of Productivity Analysis, Springer, vol. 56(2), pages 151-170, December.
    10. Wanke, Peter & Tsionas, Mike G. & Chen, Zhongfei & Moreira Antunes, Jorge Junio, 2020. "Dynamic network DEA and SFA models for accounting and financial indicators with an analysis of super-efficiency in stochastic frontiers: An efficiency comparison in OECD banking," International Review of Economics & Finance, Elsevier, vol. 69(C), pages 456-468.
    11. Yong Tan & Peter Wanke & Jorge Antunes & Ali Emrouznejad, 2021. "Unveiling endogeneity between competition and efficiency in Chinese banks: a two-stage network DEA and regression analysis," Annals of Operations Research, Springer, vol. 306(1), pages 131-171, November.
    12. Christopoulos, Apostolos G. & Dokas, Ioannis G. & Katsimardou, Sofia & Spyromitros, Eleftherios, 2020. "Assessing banking sectors’ efficiency of financially troubled Eurozone countries," Research in International Business and Finance, Elsevier, vol. 52(C).
    13. Mai, Nhat Chi, 2015. "Efficiency of the banking system in Vietnam under financial liberalization," OSF Preprints qsf6d, Center for Open Science.
    14. Adriana Cobas & Alexandros Maziotis & Andrés Villegas, 2023. "Measurement of Efficiency and its Drivers in the Chilean Banking Industry," Working Papers Central Bank of Chile 987, Central Bank of Chile.
    15. Md. Asif Nawaz, 2021. "Impact of Specialization, Ownership Structure, and Size on Cost and Profit Efficiency of US Commercial and Savings Banks," Journal of Applied Finance & Banking, SCIENPRESS Ltd, vol. 11(3), pages 1-4.
    16. Ather Hassan Dar & Somesh Kumar Mathur & Sila Mishra, 2021. "The Efficiency of Indian Banks: A DEA, Malmquist and SFA Analysis with Bad Output," Journal of Quantitative Economics, Springer;The Indian Econometric Society (TIES), vol. 19(4), pages 653-701, December.
    17. Zhao, Shirong, 2021. "Quantile estimation of stochastic frontier models with the normal–half normal specification: A cumulative distribution function approach," Economics Letters, Elsevier, vol. 206(C).
    18. Carmelo Algeri & Luc Anselin & Antonio Fabio Forgione & Carlo Migliardo, 2022. "Spatial dependence in the technical efficiency of local banks," Papers in Regional Science, Wiley Blackwell, vol. 101(3), pages 685-716, June.
    19. Jradi, Samah & Parmeter, Christopher F. & Ruggiero, John, 2021. "Quantile estimation of stochastic frontiers with the normal-exponential specification," European Journal of Operational Research, Elsevier, vol. 295(2), pages 475-483.
    20. Li, Xingchen & Xu, Guangcheng & Wu, Jie & Xu, Chengzhen & Zhu, Qingyuan, 2024. "Evaluation of bank efficiency by considering the uncertainty of nonperforming loans," Omega, Elsevier, vol. 126(C).

    More about this item

    Keywords

    banking efficiency; banking regulation; nonparametric frontier estimation; stochastic efficiency fron-tier; banking ecosystem; literature review.;
    All these keywords.

    JEL classification:

    • G21 - Financial Economics - - Financial Institutions and Services - - - Banks; Other Depository Institutions; Micro Finance Institutions; Mortgages
    • G28 - Financial Economics - - Financial Institutions and Services - - - Government Policy and Regulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aiy:jnjaer:v:20:y:2021:i:2:p:294-326. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Natalia Starodubets (email available below). General contact details of provider: https://edirc.repec.org/data/seurfru.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.