IDEAS home Printed from https://ideas.repec.org/p/idb/spdwps/1104.html
   My bibliography  Save this paper

Evaluating the Impact of Science, Technology and Innovation Programs: a Methodological Toolkit

Author

Listed:
  • Gustavo A. Crespi
  • Alessandro Maffioli
  • Pierre Mohnen
  • Gonzalo Vázquez

Abstract

The purpose of this guideline is to provide ideas and technical advice on how to measure the effectiveness of Science, Technology and Innovation Programs (STIP). The paper addresses the specific challenges of evaluating STIP, from the assessment of the intervention logic to the choice of the most appropriate method to solve the attribution problem. Much attention is devoted to the topic of data, discussing pros and cons of different data sources, data quality issues, and strategies for data collection. The paper analyzes in detail the potential application of experimental and quasi-experimental methods to STIP. For each method, the paper highlights characteristics and assumptions, practical issues related to the implementation, and strengths and weakness specifically related to the application to STIP. Other specific issues related to the evaluation of STIP are also considered: the timing of effects, intensity of treatment, multiple treatments, impact heterogeneity, externalities, and general equilibrium effects. Concrete examples of rigorous evaluations of STIP support the discussion of the various topics throughout the guideline.

Suggested Citation

  • Gustavo A. Crespi & Alessandro Maffioli & Pierre Mohnen & Gonzalo Vázquez, 2011. "Evaluating the Impact of Science, Technology and Innovation Programs: a Methodological Toolkit," SPD Working Papers 1104, Inter-American Development Bank, Office of Strategic Planning and Development Effectiveness (SPD).
    • Handle: RePEc:idb:spdwps:1104
    as

    Download full text from publisher

    File URL: http://www.iadb.org/document.cfm?id=36693990
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Richard K. Crump & V. Joseph Hotz & Guido W. Imbens & Oscar A. Mitnik, 2009. "Dealing with limited overlap in estimation of average treatment effects," Biometrika, Biometrika Trust, vol. 96(1), pages 187-199.
    2. Bresnahan, Timothy F. & Trajtenberg, M., 1995. "General purpose technologies 'Engines of growth'?," Journal of Econometrics, Elsevier, vol. 65(1), pages 83-108, January.
    3. José Miguel Benavente & Gustavo Crespi & Alessandro Maffioli, 2007. "Public Support to Firm-Level Innovation: An Evaluation of the FONTEC Program," OVE Working Papers 0507, Inter-American Development Bank, Office of Evaluation and Oversight (OVE).
    4. Raffaello Bronzini & Eleonora Iachini, 2014. "Are Incentives for R&D Effective? Evidence from a Regression Discontinuity Approach," American Economic Journal: Economic Policy, American Economic Association, vol. 6(4), pages 100-134, November.
    5. Charles Bérubé & Pierre Mohnen, 2009. "Are firms that receive R&D subsidies more innovative?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 42(1), pages 206-225, February.
    6. Bronwyn Hall & Alessandro Maffioli, 2008. "Evaluating the impact of technology development funds in emerging economies: evidence from Latin America," The European Journal of Development Research, Taylor and Francis Journals, vol. 20(2), pages 172-198.
    7. Sergio Firpo, 2007. "Efficient Semiparametric Estimation of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 75(1), pages 259-276, January.
    8. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    9. James J. Heckman & Hidehiko Ichimura & Petra E. Todd, 1997. "Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 605-654.
    10. David S. Lee & Thomas Lemieux, 2010. "Regression Discontinuity Designs in Economics," Journal of Economic Literature, American Economic Association, vol. 48(2), pages 281-355, June.
    11. Carolyn Heinrich & Alessandro Maffioli & Gonzalo Vázquez, 2010. "A Primer for Applying Propensity-Score Matching," SPD Working Papers 1005, Inter-American Development Bank, Office of Strategic Planning and Development Effectiveness (SPD).
    12. Ashenfelter, Orley C, 1978. "Estimating the Effect of Training Programs on Earnings," The Review of Economics and Statistics, MIT Press, vol. 60(1), pages 47-57, February.
    13. Markus Frölich & Blaise Melly, 2013. "Unconditional Quantile Treatment Effects Under Endogeneity," Journal of Business & Economic Statistics, Taylor & Francis Journals, vol. 31(3), pages 346-357, July.
    14. Bronwyn Hall & Alessandro Maffioli, 2008. "Evaluating the impact of technology development funds in emerging economies: evidence from Latin America," The European Journal of Development Research, Taylor and Francis Journals, vol. 20(2), pages 172-198.
    15. Markus Frlich, 2004. "Finite-Sample Properties of Propensity-Score Matching and Weighting Estimators," The Review of Economics and Statistics, MIT Press, vol. 86(1), pages 77-90, February.
    16. James J. Heckman & Jeffrey Smith & Nancy Clements, 1997. "Making The Most Out Of Programme Evaluations and Social Experiments: Accounting For Heterogeneity in Programme Impacts," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 64(4), pages 487-535.
    17. Harding, Matthew & Lamarche, Carlos, 2009. "A quantile regression approach for estimating panel data models using instrumental variables," Economics Letters, Elsevier, vol. 104(3), pages 133-135, September.
    18. Blanes, J. Vicente & Busom, Isabel, 2004. "Who participates in R&D subsidy programs?: The case of Spanish manufacturing firms," Research Policy, Elsevier, vol. 33(10), pages 1459-1476, December.
    19. Josh Lerner, 2002. "When Bureaucrats Meet Entrepreneurs: The Design of Effective "Public Venture Capital" Programmes," Economic Journal, Royal Economic Society, vol. 112(477), pages 73-84, February.
    20. Marco Caliendo & Sabine Kopeinig, 2008. "Some Practical Guidance For The Implementation Of Propensity Score Matching," Journal of Economic Surveys, Wiley Blackwell, vol. 22(1), pages 31-72, February.
    21. Boris Lokshin & Pierre Mohnen, 2012. "How effective are level-based R&D tax credits? Evidence from the Netherlands," Applied Economics, Taylor & Francis Journals, vol. 44(12), pages 1527-1538, April.
    22. Bruno Van Pottelsberghe & Steve Nysten & Esmeralda Megally, 2003. "Evaluation of current fiscal incentives for business R&D in Belgium," Working Papers CEB 03-011.RS, ULB -- Universite Libre de Bruxelles.
    23. Guido Imbens & Karthik Kalyanaraman, 2012. "Optimal Bandwidth Choice for the Regression Discontinuity Estimator," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 79(3), pages 933-959.
    24. Reiss, Peter C. & Wolak, Frank A., 2007. "Structural Econometric Modeling: Rationales and Examples from Industrial Organization," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 6, chapter 64, Elsevier.
    25. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    26. Jacob, Brian A. & Lefgren, Lars, 2011. "The impact of research grant funding on scientific productivity," Journal of Public Economics, Elsevier, vol. 95(9), pages 1168-1177.
    27. Abhijit V. Banerjee & Esther Duflo, 2009. "The Experimental Approach to Development Economics," Annual Review of Economics, Annual Reviews, vol. 1(1), pages 151-178, May.
    28. Ubfal, Diego & Maffioli, Alessandro, 2010. "The Impact of Funding on Research Collaboration: Evidence from Argentina," IDB Publications (Working Papers) 2663, Inter-American Development Bank.
    29. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    30. M. Angelucci & V. Di Maro, 2016. "Programme evaluation and spillover effects," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 8(1), pages 22-43, March.
    31. Stephen Bond, 2002. "Dynamic panel data models: a guide to microdata methods and practice," CeMMAP working papers CWP09/02, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    32. Almus, Matthias & Czarnitzki, Dirk, 2003. "The Effects of Public R&D Subsidies on Firms' Innovation Activities: The Case of Eastern Germany," Journal of Business & Economic Statistics, American Statistical Association, vol. 21(2), pages 226-236, April.
    33. Martin, Stephen & Scott, John T., 2000. "The nature of innovation market failure and the design of public support for private innovation," Research Policy, Elsevier, vol. 29(4-5), pages 437-447, April.
    34. Sebastian Galiani & Paul Gertler & Ernesto Schargrodsky, 2005. "Water for Life: The Impact of the Privatization of Water Services on Child Mortality," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 83-120, February.
    35. Soete, Luc & Verspagen, Bart & ter Weel, Bas, 2010. "Systems of Innovation," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 2, chapter 0, pages 1159-1180, Elsevier.
    36. Koenker, Roger W & Bassett, Gilbert, Jr, 1978. "Regression Quantiles," Econometrica, Econometric Society, vol. 46(1), pages 33-50, January.
    37. Zvi Griliches, 1998. "Issues in Assessing the Contribution of Research and Development to Productivity Growth," NBER Chapters, in: R&D and Productivity: The Econometric Evidence, pages 17-45, National Bureau of Economic Research, Inc.
    38. Sudipto Bhattacharya & Jay R. Ritter, 1983. "Innovation and Communication: Signalling with Partial Disclosure," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 50(2), pages 331-346.
    39. Cohen, Wesley M & Levinthal, Daniel A, 1989. "Innovation and Learning: The Two Faces of R&D," Economic Journal, Royal Economic Society, vol. 99(397), pages 569-596, September.
    40. Victor Chernozhukov & Christian Hansen, 2005. "An IV Model of Quantile Treatment Effects," Econometrica, Econometric Society, vol. 73(1), pages 245-261, January.
    41. Stiglitz, Joseph E & Weiss, Andrew, 1981. "Credit Rationing in Markets with Imperfect Information," American Economic Review, American Economic Association, vol. 71(3), pages 393-410, June.
    42. Joshua D. Angrist & Alan B. Keueger, 1991. "Does Compulsory School Attendance Affect Schooling and Earnings?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 106(4), pages 979-1014.
    43. Lerner, Josh, 1999. "The Government as Venture Capitalist: The Long-Run Impact of the SBIR Program," The Journal of Business, University of Chicago Press, vol. 72(3), pages 285-318, July.
    44. Czarnitzki, Dirk, 2002. "Research and development: financial constraints and the role of public funding for small and medium-sized enterprises," ZEW Discussion Papers 02-74, ZEW - Leibniz Centre for European Economic Research.
    45. OA Carboni, 2008. "The Effect of R&D Subsidies on Private R&D: Evidence from Italian Manufacturing Data," Working Paper CRENoS 200815, Centre for North South Economic Research, University of Cagliari and Sassari, Sardinia.
    46. Heshmati, Almas & Loof, Hans, 2005. "The Impact of Public Funds on Private R&D Investment: New Evidence from a Firm Level Innovation Study," Discussion Papers 11862, MTT Agrifood Research Finland.
    47. Diego Ubfal & Alessandro Maffioli, 2010. "The Impact of Funding on Research Collaboration: Evidence from Argentina," IDB Publications (Working Papers) 9395, Inter-American Development Bank.
    48. Nelson, Richard R, 1981. "Research on Productivity Growth and Productivity Differences: Dead Ends and New Departures," Journal of Economic Literature, American Economic Association, vol. 19(3), pages 1029-1064, September.
    49. Hall, Bronwyn & Van Reenen, John, 2000. "How effective are fiscal incentives for R&D? A review of the evidence," Research Policy, Elsevier, vol. 29(4-5), pages 449-469, April.
    50. Editors The, 2007. "From the Editors," Basic Income Studies, De Gruyter, vol. 2(1), pages 1-5, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sanginés, Mario & Prats Cabrera, Joan Oriol & Pimenta, Carlos & García Osío, Gustavo & Martínez Guzman, Juan Pablo & Marcel, Mario & Farias, Pedro & Crespi, Gustavo & Chrisney, Martin D. & Corbacho, A, 2013. "The Fiscal Institutions of Tomorrow," IDB Publications (Books), Inter-American Development Bank, number 460, December.
    2. Margaret Dalziel, 2018. "Why are there (almost) no randomised controlled trial-based evaluations of business support programmes?," Palgrave Communications, Palgrave Macmillan, vol. 4(1), pages 1-9, December.
    3. Crespi, Gustavo & Figal Garone, Lucas & Maffioli, Alessandro & Stein, Ernesto H., 2019. "Innovation, Productivity, and Spillover Effects: Evidence from Chile," IDB Publications (Working Papers) 9464, Inter-American Development Bank.
    4. Crespi, Gustavo & Figal Garone, Lucas & Maffioli, Alessandro & Stein, Ernesto, 2020. "Public support to R&D, productivity, and spillover effects: Firm-level evidence from Chile," World Development, Elsevier, vol. 130(C).
    5. Casaburi, Gabriel & Crespi, Gustavo & De León, Ignacio L. & Figal Garone, Lucas & Grazzi, Matteo & Guaipatín, Carlos & Katz, Jorge & Llisterri, Juan J. & Maffioli, Alessandro & Navarro, Juan Carlos & , 2016. "La política de innovación en América Latina y el Caribe: Nuevos caminos," IDB Publications (Books), Inter-American Development Bank, number 7705, December.
    6. Giuliani, Elisa & Maffioli, Alessandro & Pacheco, Manuel & Pietrobelli, Carlo & Stucchi, Rodolfo, 2014. "Evaluating the Impact of Cluster Development Programs," Papers in Innovation Studies 2014/10, Lund University, CIRCLE - Centre for Innovation Research.
    7. Mardones, Cristian & Velásquez, Andrés, 2021. "Macroeconomic, intersectoral, and environmental effects of R&D subsidies in Chile: An input-output approach," Technological Forecasting and Social Change, Elsevier, vol. 173(C).
    8. Aboal, Diego & Perera, Marcelo & Tacsir, Ezequiel & Vairo, Maren, 2018. "A guide for the evaluation of programs of human capital training for science, technology and innovation," MERIT Working Papers 2018-031, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    9. Dezhina, I. & Simachev, Yu., 2013. "Matching Grants for Stimulating Partnerships between Companies and Universities in Innovation Area: Initial Effects in Russia," Journal of the New Economic Association, New Economic Association, vol. 19(3), pages 99-122.
    10. Gabriel Casaburi & Gustavo Crespi & Ignacio L. De León & José Fernández & Lucas Figal Garone & Matteo Grazzi & Carlos Guaipatín & Jorge Katz & Juan J. Llisterri & Alessandro Maffioli & Juan Carlos Nav, 2016. "La política de innovación en América Latina y el Caribe: Nuevos caminos," IDB Publications (Books), Inter-American Development Bank, number 94816 edited by Juan Carlos Navarro & Jocelyn Olivari, February.
    11. Alessandro Maffioli & Diego Ubfal & Gonzalo Vazquez-Bare & Pedro Cerdan-Infantes, 2013. "Improving technology adoption in agriculture through extension services: evidence from Uruguay," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 5(1), pages 64-81, March.
    12. Mariano Pereira & Diana Suárez, 2018. "Matthew effect, capabilities and innovation policy: the Argentinean case," Economics of Innovation and New Technology, Taylor & Francis Journals, vol. 27(1), pages 62-79, January.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Guido W. Imbens & Jeffrey M. Wooldridge, 2009. "Recent Developments in the Econometrics of Program Evaluation," Journal of Economic Literature, American Economic Association, vol. 47(1), pages 5-86, March.
    2. Mario Sanginés & Joan Oriol Prats Cabrera & Carlos Pimenta & Gustavo García Osío & Juan Pablo Martínez Guzman & Mario Marcel & Pedro Farias & Gustavo Crespi & Martin D. Chrisney & Ana Corbacho, 2013. "The Fiscal Institutions of Tomorrow," IDB Publications (Books), Inter-American Development Bank, number 82298 edited by Ana Corbacho, February.
    3. Sanginés, Mario & Prats Cabrera, Joan Oriol & Pimenta, Carlos & García Osío, Gustavo & Martínez Guzman, Juan Pablo & Marcel, Mario & Farias, Pedro & Crespi, Gustavo & Chrisney, Martin D. & Corbacho, A, 2013. "The Fiscal Institutions of Tomorrow," IDB Publications (Books), Inter-American Development Bank, number 460, December.
    4. repec:idb:brikps:460 is not listed on IDEAS
    5. Czarnitzki, Dirk & Hanel, Petr & Rosa, Julio Miguel, 2011. "Evaluating the impact of R&D tax credits on innovation: A microeconometric study on Canadian firms," Research Policy, Elsevier, vol. 40(2), pages 217-229, March.
    6. José Ángel Zúñiga-Vicente & César Alonso-Borrego & Francisco J. Forcadell & José I. Galán, 2014. "Assessing The Effect Of Public Subsidies On Firm R&D Investment: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 28(1), pages 36-67, February.
    7. Bronzini, Raffaello & Piselli, Paolo, 2016. "The impact of R&D subsidies on firm innovation," Research Policy, Elsevier, vol. 45(2), pages 442-457.
    8. Clausen, Tommy H., 2009. "Do subsidies have positive impacts on R&D and innovation activities at the firm level?," Structural Change and Economic Dynamics, Elsevier, vol. 20(4), pages 239-253, December.
    9. Volpe Martincus, Christian & Carballo, Jerónimo, 2010. "Beyond the average effects: The distributional impacts of export promotion programs in developing countries," Journal of Development Economics, Elsevier, vol. 92(2), pages 201-214, July.
    10. Liao, Wen-Chi & Zhao, Daxuan, 2019. "The selection and quantile treatment effects on the economic returns of green buildings," Regional Science and Urban Economics, Elsevier, vol. 74(C), pages 38-48.
    11. Jones A.M & Rice N, 2009. "Econometric Evaluation of Health Policies," Health, Econometrics and Data Group (HEDG) Working Papers 09/09, HEDG, c/o Department of Economics, University of York.
    12. Mulugeta S. Kahsai & Randall Jackson, 2015. "Quasi-Experimental Methods an Annotated Bibliography," Working Papers Resource Document 2015-01, Regional Research Institute, West Virginia University.
    13. Volpe Martincus, Christian & Carballo, Jerónimo, 2010. "Beyond the average effects: The distributional impacts of export promotion programs in developing countries," Journal of Development Economics, Elsevier, vol. 92(2), pages 201-214, July.
    14. Wang, Yanbo & Li, Jizhen & Furman, Jeffrey L., 2017. "Firm performance and state innovation funding: Evidence from China’s innofund program," Research Policy, Elsevier, vol. 46(6), pages 1142-1161.
    15. Casaburi, Gabriel & Crespi, Gustavo & De León, Ignacio L. & Figal Garone, Lucas & Grazzi, Matteo & Guaipatín, Carlos & Katz, Jorge & Llisterri, Juan J. & Maffioli, Alessandro & Navarro, Juan Carlos & , 2016. "La política de innovación en América Latina y el Caribe: Nuevos caminos," IDB Publications (Books), Inter-American Development Bank, number 7705, December.
    16. Spyros Arvanitis, 2013. "Micro-econometric approaches to the evaluation of technology-oriented public programmes: a non-technical review of the state of the art," Chapters, in: Albert N. Link & Nicholas S. Vonortas (ed.), Handbook on the Theory and Practice of Program Evaluation, chapter 3, pages 56-88, Edward Elgar Publishing.
    17. Gabriel Casaburi & Gustavo Crespi & Ignacio L. De León & José Fernández & Lucas Figal Garone & Matteo Grazzi & Carlos Guaipatín & Jorge Katz & Juan J. Llisterri & Alessandro Maffioli & Juan Carlos Nav, 2016. "La política de innovación en América Latina y el Caribe: Nuevos caminos," IDB Publications (Books), Inter-American Development Bank, number 94816 edited by Juan Carlos Navarro & Jocelyn Olivari, February.
    18. Victor Chernozhukov & Iván Fernández‐Val & Blaise Melly, 2013. "Inference on Counterfactual Distributions," Econometrica, Econometric Society, vol. 81(6), pages 2205-2268, November.
    19. Mellace, Giovanni & Ventura, Marco, 2019. "Intended and unintended effects of public incentives for innovation. Quasi-experimental evidence from Italy," Discussion Papers on Economics 9/2019, University of Southern Denmark, Department of Economics.
    20. Hall, Bronwyn H. & Lerner, Josh, 2010. "The Financing of R&D and Innovation," Handbook of the Economics of Innovation, in: Bronwyn H. Hall & Nathan Rosenberg (ed.), Handbook of the Economics of Innovation, edition 1, volume 1, chapter 0, pages 609-639, Elsevier.
    21. Jeffrey Smith & Arthur Sweetman, 2016. "Viewpoint: Estimating the causal effects of policies and programs," Canadian Journal of Economics, Canadian Economics Association, vol. 49(3), pages 871-905, August.

    More about this item

    Keywords

    Impact Evaluation; Science; Technology Innovation; Development Effectiveness;
    All these keywords.

    JEL classification:

    • H43 - Public Economics - - Publicly Provided Goods - - - Project Evaluation; Social Discount Rate
    • O12 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Microeconomic Analyses of Economic Development
    • O14 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development - - - Industrialization; Manufacturing and Service Industries; Choice of Technology
    • O22 - Economic Development, Innovation, Technological Change, and Growth - - Development Planning and Policy - - - Project Analysis
    • O31 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Innovation and Invention: Processes and Incentives
    • O32 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Management of Technological Innovation and R&D
    • O38 - Economic Development, Innovation, Technological Change, and Growth - - Innovation; Research and Development; Technological Change; Intellectual Property Rights - - - Government Policy

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:idb:spdwps:1104. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Felipe Herrera Library (email available below). General contact details of provider: https://edirc.repec.org/data/iadbbus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.