IDEAS home Printed from https://ideas.repec.org/a/sae/intare/v22y2019i2p95-127.html
   My bibliography  Save this article

How (not) to measure democracy

Author

Listed:
  • Vanessa A Boese

Abstract

Measures of democracy are regularly employed in the statistical analysis of economic, political, and social policy. This paper reviews the measures" setup, strength, and weaknesses across the three most prominent democracy datasets: PolityIV, Freedom House, and Varieties of Democracy. The measures developed by the Varieties of Democracy project outperform Polity2 and Freedom House Index with respect to the underlying definition and measurement scale, as well as the theoretical justification of the aggregation procedure. The three indices display a high level of agreement for those observations included in all three datasets. The most substantial differences between the indices lie in the indices’ coverage, i.e. in their non-missing observations (in Polity2 coding, for example, years during which a country is occupied by foreign powers constitute missing values), the availability of disaggregate data and the above mentioned key areas. This paper clarifies when to proceed with caution, but for the most part advocates the use of Varieties of Democracy in the statistical analysis of democracy.

Suggested Citation

  • Vanessa A Boese, 2019. "How (not) to measure democracy," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 22(2), pages 95-127, June.
    • Handle: RePEc:sae:intare:v:22:y:2019:i:2:p:95-127
    DOI: 10.1177/2233865918815571
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2233865918815571
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2233865918815571?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Vanessa A. Boese & Katrin Kamin, 2019. "‘Tis but thy name that is my enemy: On the construction of macro panel datasets in conflict and peace economics," Economics of Peace and Security Journal, EPS Publishing, vol. 14(1), pages 5-26, April.
    2. Scott Gates & Håvard Hegre & Mark P. Jones & Håvard Strand, 2006. "Institutional Inconsistency and Political Instability: Polity Duration, 1800–2000," American Journal of Political Science, John Wiley & Sons, vol. 50(4), pages 893-908, October.
    3. Daron Acemoglu & Suresh Naidu & Pascual Restrepo & James A. Robinson, 2019. "Democracy Does Cause Growth," Journal of Political Economy, University of Chicago Press, vol. 127(1), pages 47-100.
    4. Eckstein, Harry, 1973. "Authority Patterns: A Structural Basis for Political Inquiry," American Political Science Review, Cambridge University Press, vol. 67(4), pages 1142-1161, December.
    5. Gurr, Ted Robert, 1974. "Persistence and Change in Political Systems, 1800–1971," American Political Science Review, Cambridge University Press, vol. 68(4), pages 1482-1504, December.
    6. James Raymond Vreeland, 2008. "The Effect of Political Regime on Civil War," Journal of Conflict Resolution, Peace Science Society (International), vol. 52(3), pages 401-425, June.
    7. Pemstein, Daniel & Meserve, Stephen A. & Melton, James, 2010. "Democratic Compromise: A Latent Variable Analysis of Ten Measures of Regime Type," Political Analysis, Cambridge University Press, vol. 18(4), pages 426-449.
    8. Przeworski,Adam & Alvarez,Michael E. & Cheibub,Jose Antonio & Limongi,Fernando, 2000. "Democracy and Development," Cambridge Books, Cambridge University Press, number 9780521793797, January.
    9. Høyland, Bjørn & Moene, Karl & Willumsen, Fredrik, 2012. "The tyranny of international index rankings," Journal of Development Economics, Elsevier, vol. 97(1), pages 1-14.
    10. Przeworski,Adam & Alvarez,Michael E. & Cheibub,Jose Antonio & Limongi,Fernando, 2000. "Democracy and Development," Cambridge Books, Cambridge University Press, number 9780521790321, January.
    11. Fearon, James D. & Laitin, David D., 2003. "Ethnicity, Insurgency, and Civil War," American Political Science Review, Cambridge University Press, vol. 97(1), pages 75-90, February.
    12. Jack A. Goldstone & Robert H. Bates & David L. Epstein & Ted Robert Gurr & Michael B. Lustik & Monty G. Marshall & Jay Ulfelder & Mark Woodward, 2010. "A Global Model for Forecasting Political Instability," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 190-208, January.
    13. Martin Elff & Sebastian Ziaja, 2018. "Method Factors in Democracy Indicators," Politics and Governance, Cogitatio Press, vol. 6(1), pages 92-104.
    14. José Cheibub & Jennifer Gandhi & James Vreeland, 2010. "Democracy and dictatorship revisited," Public Choice, Springer, vol. 143(1), pages 67-101, April.
    15. Shawn Treier & Simon Jackman, 2008. "Democracy as a Latent Variable," American Journal of Political Science, John Wiley & Sons, vol. 52(1), pages 201-217, January.
    16. Casper, Gretchen & Tufis, Claudiu, 2003. "Correlation Versus Interchangeability: The Limited Robustness of Empirical Findings on Democracy Using Highly Correlated Data Sets," Political Analysis, Cambridge University Press, vol. 11(02), pages 196-203, March.
    17. Marquardt, Kyle L. & Pemstein, Daniel, 2018. "IRT Models for Expert-Coded Panel Data," Political Analysis, Cambridge University Press, vol. 26(4), pages 431-456, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Boese-Schlosser, Vanessa A., 2024. "Are most journalists killed in democracies?," Discussion Papers, Research Unit: Transformations of Democracy SP V 2024-501, WZB Berlin Social Science Center.
    2. Gründler, Klaus & Krieger, Tommy, 2022. "Should we care (more) about data aggregation?," European Economic Review, Elsevier, vol. 142(C).
    3. Chimere O. Iheonu & Simplice A. Asongu & Shedrach A. Agbutun & Innocent A. Ifelunini, 2021. "Democracy and Terrorism in Africa," Working Papers of the African Governance and Development Institute. 21/089, African Governance and Development Institute..
    4. Bergougui, Brahim & Murshed, Syed Mansoob, 2020. "New evidence on the oil-democracy nexus utilising the Varieties of Democracy data," Resources Policy, Elsevier, vol. 69(C).
    5. Campomanes, Ignacio P., 2024. "The political economy of inequality, mobility and redistribution," Journal of Macroeconomics, Elsevier, vol. 79(C).
    6. repec:aly:journl:202192 is not listed on IDEAS
    7. Henri Njangang & Youssouf Nvuh-Njoya, 2023. "Unravelling the link between democracy and economic complexity: fresh evidence from the Varieties of Democracy data," SN Business & Economics, Springer, vol. 3(3), pages 1-32, March.
    8. Seraphine F. Maerz & Carsten Q. Schneider, 2020. "Comparing public communication in democracies and autocracies: automated text analyses of speeches by heads of government," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 517-545, April.
    9. Klaus Gründler & Tommy Krieger, 2021. "Using Machine Learning for Measuring Democracy: An Update," CESifo Working Paper Series 8903, CESifo.
    10. Gründler, Klaus & Krieger, Tommy, 2021. "Using Machine Learning for measuring democracy: A practitioners guide and a new updated dataset for 186 countries from 1919 to 2019," European Journal of Political Economy, Elsevier, vol. 70(C).
    11. Vanessa Boese-Schlosser & Markus Eberhardt, 2023. "How does democracy cause growth?," Discussion Papers 2023-13, Nottingham Interdisciplinary Centre for Economic and Political Research (NICEP).
    12. Andrea Vaccaro, 2022. "An empirical evaluation of common cross-national measures of state capacity," RIEDS - Rivista Italiana di Economia, Demografia e Statistica - The Italian Journal of Economic, Demographic and Statistical Studies, SIEDS Societa' Italiana di Economia Demografia e Statistica, vol. 76(1), pages 157-168, January-M.
    13. Debarati Basu & Shabana Mitra & Archana Purohit, 2023. "Measuring Partial Democracies: Rules and their Implementation," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 166(1), pages 133-155, February.
    14. Michaela Vourvoulia & Athanasios Kampas, 2024. "Are democratic regime and the magnitude of the informal economy robust determinants of human impacts on the environment? An extreme bounds analysis," Economics and Politics, Wiley Blackwell, vol. 36(1), pages 611-629, March.
    15. Jaap W.B. Bos & Jasmin Gröschl & Martien Lamers & Runliang Li & Mark Sanders & Vincent Schippers & Jasmin Katrin Gröschl, 2022. "How Do Institutions Affect the Impact of Natural Disasters?," CESifo Working Paper Series 10174, CESifo.
    16. Wafa Ghardallou, 2022. "Financial System Development and Democracy: a Panel Smooth Transition Regression Approach for Developing Countries," Journal of the Knowledge Economy, Springer;Portland International Center for Management of Engineering and Technology (PICMET), vol. 13(2), pages 1714-1735, June.
    17. Annaka, Susumu & Higashijima, Masaaki, 2021. "Political liberalization and human development: Dynamic effects of political regime change on infant mortality across three centuries (1800-2015)," World Development, Elsevier, vol. 147(C).
    18. Hendrix, Cullen & Noland, Marcus, 2023. "The West versus Beijing? Determinants of the UN Human Rights Council vote (not) to debate human rights in Xinjiang," MPRA Paper 118630, University Library of Munich, Germany.
    19. Ginevra Peruginelli & Janne Pölönen, 2024. "The legal foundation of responsible research assessment: An overview on European Union and Italy," Research Evaluation, Oxford University Press, vol. 32(4), pages 670-682.
    20. Kaiya Wu & Shiping Tang & Min Tang, 2024. "Interpretative structural modeling to social sciences: designing better datasets for mixed method research," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(5), pages 4073-4092, October.
    21. Lars Pelke, 2023. "Reanalysing the link between democracy and economic development," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 26(4), pages 361-383, December.
    22. Godfred Bonnah Nkansah, 2022. "Youth Cohort Size, Structural Socioeconomic Conditions, and Youth Protest Behavior in Democratic Societies (1995–2014)," SAGE Open, , vol. 12(2), pages 21582440221, May.
    23. Nuno Garoupa & Rok Spruk, 2024. "Measuring Political Institutions in the Long Run: A Latent Variable Analysis of Political Regimes, 1810–2018," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 867-914, July.
    24. Jamie Marie Sommer & Michael Restivo & John M. Shandra, 2024. "Corruption and Palm Oil in a Cross-National Perspective: How India Contributes to Forest Loss in Peripheral Nations," Journal of South Asian Development, , vol. 19(1), pages 83-111, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giebler, Heiko & Ruth, Saskia P. & Tanneberg, Dag, 2018. "Why Choice Matters: Revisiting and Comparing Measures of Democracy," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 6(1), pages 1-10.
    2. Heiko Giebler & Saskia P. Ruth & Dag Tanneberg, 2018. "Why Choice Matters: Revisiting and Comparing Measures of Democracy," Politics and Governance, Cogitatio Press, vol. 6(1), pages 1-10.
    3. Fabio Monteforte & Jonathan R. W. Temple, 2020. "The autocratic gamble: evidence from robust variance tests," Economics of Governance, Springer, vol. 21(4), pages 363-384, December.
    4. Nuno Garoupa & Rok Spruk, 2024. "Measuring Political Institutions in the Long Run: A Latent Variable Analysis of Political Regimes, 1810–2018," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 867-914, July.
    5. Méon, Pierre-Guillaume & Sekkat, Khalid, 2022. "A time to throw stones, a time to reap: how long does it take for democratic transitions to improve institutional outcomes?," Journal of Institutional Economics, Cambridge University Press, vol. 18(3), pages 429-443, June.
    6. Sunde, Uwe & Fortunato, Piergiuseppe & Cervellati, Matteo, 2011. "Democratization and Civil Liberties: The Role of Violence During the Transition," CEPR Discussion Papers 8315, C.E.P.R. Discussion Papers.
    7. Boese Vanessa A., 2015. "Viva la Revolución, or: Do Revolutions Lead to More Democracy?," Peace Economics, Peace Science, and Public Policy, De Gruyter, vol. 21(4), pages 541-551, December.
    8. Cervellati, Matteo & Fortunato, Piergiuseppe & Sunde, Uwe, 2014. "Violence during democratization and the quality of democratic institutions," European Economic Review, Elsevier, vol. 66(C), pages 226-247.
    9. Alberto Batinti & Joan Costa‐Font & Timothy J. Hatton, 2022. "Voting Up? The Effects of Democracy and Franchise Extension on Human Stature," Economica, London School of Economics and Political Science, vol. 89(353), pages 161-190, January.
    10. Jamie Levin & Joseph MacKay & Anne Spencer Jamison & Abouzar Nasirzadeh & Anthony Sealey, 2021. "A test of the democratic peacekeeping hypothesis: Coups, democracy, and foreign military deployments," Journal of Peace Research, Peace Research Institute Oslo, vol. 58(3), pages 355-367, May.
    11. Gründler, Klaus & Krieger, Tommy, 2022. "Should we care (more) about data aggregation?," European Economic Review, Elsevier, vol. 142(C).
    12. Henrikas BartuseviÄ ius & Svend-Erik Skaaning, 2018. "Revisiting democratic civil peace," Journal of Peace Research, Peace Research Institute Oslo, vol. 55(5), pages 625-640, September.
    13. Vanessa Alexandra Boese & Matthew Charles Wilson, 2023. "Contestation and participation: Concepts, measurement, and inference," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 26(2), pages 89-106, June.
    14. Svend-Erik Skaaning, 2018. "Different Types of Data and the Validity of Democracy Measures," Politics and Governance, Cogitatio Press, vol. 6(1), pages 105-116.
    15. repec:fsu:wpaper:wp2013_12_01 is not listed on IDEAS
    16. Gabriel Leon, 2014. "Loyalty for sale? Military spending and coups d’etat," Public Choice, Springer, vol. 159(3), pages 363-383, June.
    17. Wegenast, Tim, 2010. "Inclusive Institutions and the Onset of Internal Conflict in Resource-rich Countries," GIGA Working Papers 126, GIGA German Institute of Global and Area Studies.
    18. Kammas, Pantelis & Sarantides, Vassilis, 2019. "Do dictatorships redistribute more?," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 176-195.
    19. Espen Geelmuyden Rød & Carl Henrik Knutsen & Håvard Hegre, 2020. "The determinants of democracy: a sensitivity analysis," Public Choice, Springer, vol. 185(1), pages 87-111, October.
    20. Antonio Ciccone & Adilzhan Ismailov, 2022. "Rainfall, Agricultural Output and Persistent Democratization," Economica, London School of Economics and Political Science, vol. 89(354), pages 229-257, April.
    21. Kotera, Go & Okada, Keisuke, 2017. "How does democratization affect the composition of government expenditure?," Journal of Economic Behavior & Organization, Elsevier, vol. 137(C), pages 145-159.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:intare:v:22:y:2019:i:2:p:95-127. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: http://www.hufs.ac.kr/user/hufsenglish/re_1.jsp .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.