IDEAS home Printed from https://ideas.repec.org/p/cgd/wpaper/362.html
   My bibliography  Save this paper

We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362

Author

Listed:
  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur

Abstract

Much of the data underlying global poverty and inequality estimates is not in the public domain, but can be accessed in small pieces using the World Bank’s PovcalNet online tool. To overcome these limitations and reproduce this database in a format more useful to researchers, we ran approximately 23 million queries of the World Bank’s web site, accessing only information that was already in the public domain. This web scraping exercise produced 10,000 points on the cumulative distribution of income or consumption from each of 942 surveys spanning 127 countries over the period 1977 to 2012. This short note describes our methodology, briefly discusses some of the relevant intellectual property issues, and illustrates the kind of calculations that are facilitated by this data set, including growth incidence curves and poverty rates using alternative PPP indices. The full data can be downloaded at www.cgdev.org/povcalnet.

Suggested Citation

  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur, 2014. "We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362," Working Papers 362, Center for Global Development.
    • Handle: RePEc:cgd:wpaper:362
    as

    Download full text from publisher

    File URL: http://www.cgdev.org/sites/default/files/dykstra-sandefur-povcal-net-world-bank_1.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Branko Milanovic, 2002. "True World Income Distribution, 1988 and 1993: First Calculation Based on Household Surveys Alone," Economic Journal, Royal Economic Society, vol. 112(476), pages 51-92, January.
    2. repec:pri:rpdevs:deaton_price_indexes_inequality_and_the_measurement_of_world_poverty_aer. is not listed on IDEAS
    3. Angus Deaton, 2010. "Price indexes, inequality, and the measurement of world poverty," Working Papers 1207, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    4. Angus Deaton & Olivier Dupriez, 2011. "Purchasing Power Parity Exchange Rates for the Global Poor," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 137-166, April.
    5. repec:pri:rpdevs:presidential%20address%2017january%202010%20all is not listed on IDEAS
    6. Angus Deaton, 2010. "Price Indexes, Inequality, and the Measurement of World Poverty," American Economic Review, American Economic Association, vol. 100(1), pages 5-34, March.
    7. Datt, Gaurav, 1998. "Computational tools for poverty measurement and analysis," FCND discussion papers 50, International Food Policy Research Institute (IFPRI).
    8. Shaohua Chen & Martin Ravallion, 2010. "The Developing World is Poorer than We Thought, But No Less Successful in the Fight Against Poverty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(4), pages 1577-1625.
    9. repec:pri:rpdevs:presidential%20address%2017january%202010%20all.pdf is not listed on IDEAS
    10. Angus Deaton & Olivier Dupriez, 2009. "Purchasing power parity exchange rates for the global poor," Working Papers 1187, Princeton University, Woodrow Wilson School of Public and International Affairs, Research Program in Development Studies..
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Moatsos, Michail, 2020. "Global Absolute Poverty: The Evolution of its Measurement," EconStor Preprints 216642, ZBW - Leibniz Information Centre for Economics.
    2. Kanbur, Ravi & Ortiz-Juarez, Eduardo & Sumner, Andy, 2024. "Is the era of declining global income inequality over?," Structural Change and Economic Dynamics, Elsevier, vol. 70(C), pages 45-55.
    3. Benfica, R. & Henderson, H., 2018. "IFAD RESEARCH SERIES 23 - The effect of the sectoral composition of economic growth on rural and urban poverty," IFAD Research Series 280072, International Fund for Agricultural Development (IFAD).
    4. Haroon Bhorat & Ravi Kanbur & Benjamin Stanwix, 2017. "Minimum Wages in Sub-Saharan Africa: A Primer," The World Bank Research Observer, World Bank, vol. 32(1), pages 21-74.
    5. Moatsos, Michail & Lazopoulos, Achillefs, 2021. "Global poverty: A first estimation of its uncertainty," World Development Perspectives, Elsevier, vol. 22(C).
    6. Bolch, Kimberly B. & Ceriani, Lidia & López-Calva, Luis F., 2022. "The arithmetics and politics of domestic resource mobilization for poverty eradication," World Development, Elsevier, vol. 149(C).
    7. Moatsos, Michail, 2018. "Scrapping PovcalNet's Distributions for Poverty Research," MPRA Paper 89872, University Library of Munich, Germany.
    8. Robert Inklaar & D. S. Prasada Rao, 2017. "Cross-Country Income Levels over Time: Did the Developing World Suddenly Become Much Richer?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 265-290, January.
    9. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Facundo Alvaredo & Leonardo Gasparini, 2013. "Recent Trends in Inequality and Poverty in Developing Countries," CEDLAS, Working Papers 0151, CEDLAS, Universidad Nacional de La Plata.
    2. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.
    3. Dhongde, Shatakshee & Minoiu, Camelia, 2013. "Global Poverty Estimates: A Sensitivity Analysis," World Development, Elsevier, vol. 44(C), pages 1-13.
    4. Robert C. Allen, 2017. "Absolute Poverty: When Necessity Displaces Desire," American Economic Review, American Economic Association, vol. 107(12), pages 3690-3721, December.
    5. Prize Committee, Nobel, 2015. "Consumption, Poverty, and Welfare," Nobel Prize in Economics documents 2015-2, Nobel Prize Committee.
    6. Peter Edward & Andy Sumner, 2015. "New estimates of global poverty and inequality: How much difference do price data," Working Papers 365, ECINEQ, Society for the Study of Economic Inequality.
    7. Robert Inklaar & D. S. Prasada Rao, 2017. "Cross-Country Income Levels over Time: Did the Developing World Suddenly Become Much Richer?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 265-290, January.
    8. Crombrugghe, D.P.I. de & Szirmai, A. & Bluhm, R., 2014. "Poor trends: The pace of poverty reduction after the Millennium Development Agenda," MERIT Working Papers 2014-006, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    9. John Gibson, 2016. "Poverty Measurement: We Know Less than Policy Makers Realize," Asia and the Pacific Policy Studies, Wiley Blackwell, vol. 3(3), pages 430-442, September.
    10. Leonardo Lucchetti & Andrés Castañeda & Santiago Garriga & Leonardo Gasparini & Daniel Valderrama, 2018. "How Sensitive Is Regional Poverty Measurement in Latin America to the Value of the Poverty Line?," Economía Journal, The Latin American and Caribbean Economic Association - LACEA, vol. 0(Fall 2018), pages 33-58, November.
    11. Cockburn, John & Duclos, Jean-Yves & Zabsonré, Agnès, 2014. "Is global social welfare increasing? A critical-level enquiry," Journal of Public Economics, Elsevier, vol. 118(C), pages 151-162.
    12. Angus Deaton & Bettina Aten, 2017. "Trying to Understand the PPPs in ICP 2011: Why Are the Results So Different?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 243-264, January.
    13. Angus Deaton, 2016. "Measuring and Understanding Behavior, Welfare, and Poverty," American Economic Review, American Economic Association, vol. 106(6), pages 1221-1243, June.
    14. Branko Milanovic, 2012. "Global inequality recalculated and updated: the effect of new PPP estimates on global inequality and 2005 estimates," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(1), pages 1-18, March.
    15. Martin Ravallion, 2016. "Toward better global poverty measures," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 14(2), pages 227-248, June.
    16. Decerf,Benoit Marie A, 2023. "An Axiomatic Study Contrasting the Two Main Poverty Line Rules," Policy Research Working Paper Series 10519, The World Bank.
    17. Duangkamon Chotikapanich & William Griffiths & Wasana Karunarathne & D.S. Prasada Rao, 2013. "Calculating Poverty Measures from the Generalised Beta Income Distribution," The Economic Record, The Economic Society of Australia, vol. 89, pages 48-66, June.
    18. Pinkovskiy, Maxim L., 2013. "World welfare is rising: Estimation using nonparametric bounds on welfare measures," Journal of Public Economics, Elsevier, vol. 97(C), pages 176-195.
    19. Alan Heston & D.S Prasada Rao, 2021. "Price Levels, Size, Distribution and Growth of the World Economy: Insights from recent International Comparisons of Prices and Real Product," CEPA Working Papers Series WP102021, School of Economics, University of Queensland, Australia.
    20. Hanjie Wang & Qiran Zhao & Yunli Bai & Linxiu Zhang & Xiaohua Yu, 2020. "Poverty and Subjective Poverty in Rural China," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 150(1), pages 219-242, July.

    More about this item

    Keywords

    poverty; inequality; consumption; income distribution; open data;
    All these keywords.

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • O57 - Economic Development, Innovation, Technological Change, and Growth - - Economywide Country Studies - - - Comparative Studies of Countries

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cgd:wpaper:362. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Publications Manager (email available below). General contact details of provider: https://edirc.repec.org/data/cgdevus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.