IDEAS home Printed from https://ideas.repec.org/p/cgd/wpaper/362.html
   My bibliography  Save this paper

We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362

Author

Listed:
  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur

Abstract

Much of the data underlying global poverty and inequality estimates is not in the public domain, but can be accessed in small pieces using the World Bank’s PovcalNet online tool. To overcome these limitations and reproduce this database in a format more useful to researchers, we ran approximately 23 million queries of the World Bank’s web site, accessing only information that was already in the public domain. This web scraping exercise produced 10,000 points on the cumulative distribution of income or consumption from each of 942 surveys spanning 127 countries over the period 1977 to 2012. This short note describes our methodology, briefly discusses some of the relevant intellectual property issues, and illustrates the kind of calculations that are facilitated by this data set, including growth incidence curves and poverty rates using alternative PPP indices. The full data can be downloaded at www.cgdev.org/povcalnet.

Suggested Citation

  • Sarah Dykstra, Benjamin Dykstra, and Justin Sandefur, 2014. "We Just Ran Twenty-Three Million Queries of the World Bank's Website - Working Paper 362," Working Papers 362, Center for Global Development.
    • Handle: RePEc:cgd:wpaper:362
    as

    Download full text from publisher

    File URL: http://www.cgdev.org/sites/default/files/dykstra-sandefur-povcal-net-world-bank_1.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. repec:pri:rpdevs:deaton_price_indexes_inequality_and_the_measurement_of_world_poverty_aer. is not listed on IDEAS
    2. Branko Milanovic, 2002. "True World Income Distribution, 1988 and 1993: First Calculation Based on Household Surveys Alone," Economic Journal, Royal Economic Society, vol. 112(476), pages 51-92, January.
    3. Angus Deaton, 2010. "Price Indexes, Inequality, and the Measurement of World Poverty," American Economic Review, American Economic Association, vol. 100(1), pages 5-34, March.
    4. Angus Deaton & Olivier Dupriez, 2011. "Purchasing Power Parity Exchange Rates for the Global Poor," American Economic Journal: Applied Economics, American Economic Association, vol. 3(2), pages 137-166, April.
    5. Datt, Gaurav, 1998. "Computational tools for poverty measurement and analysis," FCND discussion papers 50, International Food Policy Research Institute (IFPRI).
    6. Shaohua Chen & Martin Ravallion, 2010. "The Developing World is Poorer than We Thought, But No Less Successful in the Fight Against Poverty," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 125(4), pages 1577-1625.
    7. repec:pri:rpdevs:presidential%20address%2017january%202010%20all.pdf is not listed on IDEAS
    8. repec:pri:rpdevs:presidential%20address%2017january%202010%20all is not listed on IDEAS
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Moatsos, Michail, 2020. "Global Absolute Poverty: The Evolution of its Measurement," EconStor Preprints 216642, ZBW - Leibniz Information Centre for Economics.
    2. Benfica, R. & Henderson, H., 2018. "IFAD RESEARCH SERIES 23 - The effect of the sectoral composition of economic growth on rural and urban poverty," IFAD Research Series 280072, International Fund for Agricultural Development (IFAD).
    3. Haroon Bhorat & Ravi Kanbur & Benjamin Stanwix, 2017. "Minimum Wages in Sub-Saharan Africa: A Primer," The World Bank Research Observer, World Bank, vol. 32(1), pages 21-74.
    4. Moatsos, Michail & Lazopoulos, Achillefs, 2021. "Global poverty: A first estimation of its uncertainty," World Development Perspectives, Elsevier, vol. 22(C).
    5. Bolch, Kimberly B. & Ceriani, Lidia & López-Calva, Luis F., 2022. "The arithmetics and politics of domestic resource mobilization for poverty eradication," World Development, Elsevier, vol. 149(C).
    6. Moatsos, Michail, 2018. "Scrapping PovcalNet's Distributions for Poverty Research," MPRA Paper 89872, University Library of Munich, Germany.
    7. Robert Inklaar & D. S. Prasada Rao, 2017. "Cross-Country Income Levels over Time: Did the Developing World Suddenly Become Much Richer?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 265-290, January.
    8. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Facundo Alvaredo & Leonardo Gasparini, 2013. "Recent Trends in Inequality and Poverty in Developing Countries," CEDLAS, Working Papers 0151, CEDLAS, Universidad Nacional de La Plata.
    2. Pinkovskiy, Maxim L., 2013. "World welfare is rising: Estimation using nonparametric bounds on welfare measures," Journal of Public Economics, Elsevier, vol. 97(C), pages 176-195.
    3. Camelia Minoiu & Sanjay Reddy, 2014. "Kernel density estimation on grouped data: the case of poverty assessment," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 12(2), pages 163-189, June.
    4. Alan Heston & D.S. Prasada Rao, 2021. "Price Levels, Size, Distribution and Growth of the World Economy: Insights from recent International Comparisons of Prices and Real Product," PIER Working Paper Archive 21-013, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania.
    5. Cockburn, John & Duclos, Jean-Yves & Zabsonré, Agnès, 2014. "Is global social welfare increasing? A critical-level enquiry," Journal of Public Economics, Elsevier, vol. 118(C), pages 151-162.
    6. Moatsos Michail, 2016. "Global Absolute Poverty: Behind the Veil of Dollars," Journal of Globalization and Development, De Gruyter, vol. 7(2), pages 1-28, December.
    7. Crombrugghe, D.P.I. de & Szirmai, A. & Bluhm, R., 2014. "Poor trends: The pace of poverty reduction after the Millennium Development Agenda," MERIT Working Papers 2014-006, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    8. Duangkamon Chotikapanich & William Griffiths & Wasana Karunarathne & D.S. Prasada Rao, 2013. "Calculating Poverty Measures from the Generalised Beta Income Distribution," The Economic Record, The Economic Society of Australia, vol. 89, pages 48-66, June.
    9. Branko Milanovic, 2012. "Global inequality recalculated and updated: the effect of new PPP estimates on global inequality and 2005 estimates," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 10(1), pages 1-18, March.
    10. Dhongde, Shatakshee & Minoiu, Camelia, 2013. "Global Poverty Estimates: A Sensitivity Analysis," World Development, Elsevier, vol. 44(C), pages 1-13.
    11. Robert C. Allen, 2017. "Absolute Poverty: When Necessity Displaces Desire," American Economic Review, American Economic Association, vol. 107(12), pages 3690-3721, December.
    12. Peter Edward & Andy Sumner, 2014. "The Poor, the Prosperous and the ‘Inbetweeners’: A Fresh Perspective on Global Society, Inequality and Growth," Working Papers 122, International Policy Centre for Inclusive Growth.
    13. Prize Committee, Nobel, 2015. "Consumption, Poverty, and Welfare," Nobel Prize in Economics documents 2015-2, Nobel Prize Committee.
    14. Clare Leaver & Paul Segal, 2014. "The Global Distribution of Income," Economics Series Working Papers 714, University of Oxford, Department of Economics.
    15. Segal, Paul, 2011. "Resource Rents, Redistribution, and Halving Global Poverty: The Resource Dividend," World Development, Elsevier, vol. 39(4), pages 475-489, April.
    16. Peter Edward & Andy Sumner, 2015. "New estimates of global poverty and inequality: How much difference do price data," Working Papers 365, ECINEQ, Society for the Study of Economic Inequality.
    17. Robert Inklaar & D. S. Prasada Rao, 2017. "Cross-Country Income Levels over Time: Did the Developing World Suddenly Become Much Richer?," American Economic Journal: Macroeconomics, American Economic Association, vol. 9(1), pages 265-290, January.
    18. John COCKBURN & Jean-Yves DUCLOS & Agnès ZABSONRÉ, 2011. "Is the value of humanity increasing? A critical-level enquiry," Working Papers I13, FERDI.
    19. John Gibson, 2016. "Poverty Measurement: We Know Less than Policy Makers Realize," Asia and the Pacific Policy Studies, Wiley Blackwell, vol. 3(3), pages 430-442, September.
    20. Kym Anderson & John Cockburn & Will Martin, 2011. "Would Freeing Up World Trade Reduce Poverty and Inequality? The Vexed Role of Agricultural Distortions," The World Economy, Wiley Blackwell, vol. 34(4), pages 487-515, April.

    More about this item

    Keywords

    poverty; inequality; consumption; income distribution; open data;
    All these keywords.

    JEL classification:

    • D31 - Microeconomics - - Distribution - - - Personal Income and Wealth Distribution
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty
    • O57 - Economic Development, Innovation, Technological Change, and Growth - - Economywide Country Studies - - - Comparative Studies of Countries

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cgd:wpaper:362. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Publications Manager (email available below). General contact details of provider: https://edirc.repec.org/data/cgdevus.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.