IDEAS home Printed from https://ideas.repec.org/a/sae/intare/v22y2019i2p95-127.html
   My bibliography  Save this article

How (not) to measure democracy

Author

Listed:
  • Vanessa A Boese

Abstract

Measures of democracy are regularly employed in the statistical analysis of economic, political, and social policy. This paper reviews the measures" setup, strength, and weaknesses across the three most prominent democracy datasets: PolityIV, Freedom House, and Varieties of Democracy. The measures developed by the Varieties of Democracy project outperform Polity2 and Freedom House Index with respect to the underlying definition and measurement scale, as well as the theoretical justification of the aggregation procedure. The three indices display a high level of agreement for those observations included in all three datasets. The most substantial differences between the indices lie in the indices’ coverage, i.e. in their non-missing observations (in Polity2 coding, for example, years during which a country is occupied by foreign powers constitute missing values), the availability of disaggregate data and the above mentioned key areas. This paper clarifies when to proceed with caution, but for the most part advocates the use of Varieties of Democracy in the statistical analysis of democracy.

Suggested Citation

  • Vanessa A Boese, 2019. "How (not) to measure democracy," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 22(2), pages 95-127, June.
    • Handle: RePEc:sae:intare:v:22:y:2019:i:2:p:95-127
    DOI: 10.1177/2233865918815571
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/2233865918815571
    Download Restriction: no
    File URL: https://libkey.io/10.1177/2233865918815571?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Vanessa A. Boese & Katrin Kamin, 2019. "‘Tis but thy name that is my enemy: On the construction of macro panel datasets in conflict and peace economics," Economics of Peace and Security Journal, EPS Publishing, vol. 14(1), pages 5-26, April.
    2. Scott Gates & Håvard Hegre & Mark P. Jones & Håvard Strand, 2006. "Institutional Inconsistency and Political Instability: Polity Duration, 1800–2000," American Journal of Political Science, John Wiley & Sons, vol. 50(4), pages 893-908, October.
    3. Daron Acemoglu & Suresh Naidu & Pascual Restrepo & James A. Robinson, 2019. "Democracy Does Cause Growth," Journal of Political Economy, University of Chicago Press, vol. 127(1), pages 47-100.
    4. Eckstein, Harry, 1973. "Authority Patterns: A Structural Basis for Political Inquiry," American Political Science Review, Cambridge University Press, vol. 67(4), pages 1142-1161, December.
    5. Gurr, Ted Robert, 1974. "Persistence and Change in Political Systems, 1800–1971," American Political Science Review, Cambridge University Press, vol. 68(4), pages 1482-1504, December.
    6. James Raymond Vreeland, 2008. "The Effect of Political Regime on Civil War," Journal of Conflict Resolution, Peace Science Society (International), vol. 52(3), pages 401-425, June.
    7. Pemstein, Daniel & Meserve, Stephen A. & Melton, James, 2010. "Democratic Compromise: A Latent Variable Analysis of Ten Measures of Regime Type," Political Analysis, Cambridge University Press, vol. 18(4), pages 426-449.
    8. Przeworski,Adam & Alvarez,Michael E. & Cheibub,Jose Antonio & Limongi,Fernando, 2000. "Democracy and Development," Cambridge Books, Cambridge University Press, number 9780521793797, January.
    9. Høyland, Bjørn & Moene, Karl & Willumsen, Fredrik, 2012. "The tyranny of international index rankings," Journal of Development Economics, Elsevier, vol. 97(1), pages 1-14.
    10. Przeworski,Adam & Alvarez,Michael E. & Cheibub,Jose Antonio & Limongi,Fernando, 2000. "Democracy and Development," Cambridge Books, Cambridge University Press, number 9780521790321, January.
    11. Fearon, James D. & Laitin, David D., 2003. "Ethnicity, Insurgency, and Civil War," American Political Science Review, Cambridge University Press, vol. 97(1), pages 75-90, February.
    12. Jack A. Goldstone & Robert H. Bates & David L. Epstein & Ted Robert Gurr & Michael B. Lustik & Monty G. Marshall & Jay Ulfelder & Mark Woodward, 2010. "A Global Model for Forecasting Political Instability," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 190-208, January.
    13. Martin Elff & Sebastian Ziaja, 2018. "Method Factors in Democracy Indicators," Politics and Governance, Cogitatio Press, vol. 6(1), pages 92-104.
    14. José Cheibub & Jennifer Gandhi & James Vreeland, 2010. "Democracy and dictatorship revisited," Public Choice, Springer, vol. 143(1), pages 67-101, April.
    15. Shawn Treier & Simon Jackman, 2008. "Democracy as a Latent Variable," American Journal of Political Science, John Wiley & Sons, vol. 52(1), pages 201-217, January.
    16. Casper, Gretchen & Tufis, Claudiu, 2003. "Correlation Versus Interchangeability: The Limited Robustness of Empirical Findings on Democracy Using Highly Correlated Data Sets," Political Analysis, Cambridge University Press, vol. 11(02), pages 196-203, March.
    17. Marquardt, Kyle L. & Pemstein, Daniel, 2018. "IRT Models for Expert-Coded Panel Data," Political Analysis, Cambridge University Press, vol. 26(4), pages 431-456, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Giebler, Heiko & Ruth, Saskia P. & Tanneberg, Dag, 2018. "Why Choice Matters: Revisiting and Comparing Measures of Democracy," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 6(1), pages 1-10.
    2. Heiko Giebler & Saskia P. Ruth & Dag Tanneberg, 2018. "Why Choice Matters: Revisiting and Comparing Measures of Democracy," Politics and Governance, Cogitatio Press, vol. 6(1), pages 1-10.
    3. Fabio Monteforte & Jonathan R. W. Temple, 2020. "The autocratic gamble: evidence from robust variance tests," Economics of Governance, Springer, vol. 21(4), pages 363-384, December.
    4. Nuno Garoupa & Rok Spruk, 2024. "Measuring Political Institutions in the Long Run: A Latent Variable Analysis of Political Regimes, 1810–2018," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 173(3), pages 867-914, July.
    5. Méon, Pierre-Guillaume & Sekkat, Khalid, 2022. "A time to throw stones, a time to reap: how long does it take for democratic transitions to improve institutional outcomes?," Journal of Institutional Economics, Cambridge University Press, vol. 18(3), pages 429-443, June.
    6. Sunde, Uwe & Fortunato, Piergiuseppe & Cervellati, Matteo, 2011. "Democratization and Civil Liberties: The Role of Violence During the Transition," CEPR Discussion Papers 8315, C.E.P.R. Discussion Papers.
    7. Boese Vanessa A., 2015. "Viva la Revolución, or: Do Revolutions Lead to More Democracy?," Peace Economics, Peace Science, and Public Policy, De Gruyter, vol. 21(4), pages 541-551, December.
    8. Cervellati, Matteo & Fortunato, Piergiuseppe & Sunde, Uwe, 2014. "Violence during democratization and the quality of democratic institutions," European Economic Review, Elsevier, vol. 66(C), pages 226-247.
    9. Alberto Batinti & Joan Costa‐Font & Timothy J. Hatton, 2022. "Voting Up? The Effects of Democracy and Franchise Extension on Human Stature," Economica, London School of Economics and Political Science, vol. 89(353), pages 161-190, January.
    10. Jamie Levin & Joseph MacKay & Anne Spencer Jamison & Abouzar Nasirzadeh & Anthony Sealey, 2021. "A test of the democratic peacekeeping hypothesis: Coups, democracy, and foreign military deployments," Journal of Peace Research, Peace Research Institute Oslo, vol. 58(3), pages 355-367, May.
    11. Gründler, Klaus & Krieger, Tommy, 2022. "Should we care (more) about data aggregation?," European Economic Review, Elsevier, vol. 142(C).
    12. Henrikas BartuseviÄ ius & Svend-Erik Skaaning, 2018. "Revisiting democratic civil peace," Journal of Peace Research, Peace Research Institute Oslo, vol. 55(5), pages 625-640, September.
    13. Vanessa Alexandra Boese & Matthew Charles Wilson, 2023. "Contestation and participation: Concepts, measurement, and inference," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 26(2), pages 89-106, June.
    14. Svend-Erik Skaaning, 2018. "Different Types of Data and the Validity of Democracy Measures," Politics and Governance, Cogitatio Press, vol. 6(1), pages 105-116.
    15. repec:fsu:wpaper:wp2013_12_01 is not listed on IDEAS
    16. Gabriel Leon, 2014. "Loyalty for sale? Military spending and coups d’etat," Public Choice, Springer, vol. 159(3), pages 363-383, June.
    17. Wegenast, Tim, 2010. "Inclusive Institutions and the Onset of Internal Conflict in Resource-rich Countries," GIGA Working Papers 126, GIGA German Institute of Global and Area Studies.
    18. Kammas, Pantelis & Sarantides, Vassilis, 2019. "Do dictatorships redistribute more?," Journal of Comparative Economics, Elsevier, vol. 47(1), pages 176-195.
    19. Espen Geelmuyden Rød & Carl Henrik Knutsen & Håvard Hegre, 2020. "The determinants of democracy: a sensitivity analysis," Public Choice, Springer, vol. 185(1), pages 87-111, October.
    20. Antonio Ciccone & Adilzhan Ismailov, 2022. "Rainfall, Agricultural Output and Persistent Democratization," Economica, London School of Economics and Political Science, vol. 89(354), pages 229-257, April.
    21. Kotera, Go & Okada, Keisuke, 2017. "How does democratization affect the composition of government expenditure?," Journal of Economic Behavior & Organization, Elsevier, vol. 137(C), pages 145-159.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:intare:v:22:y:2019:i:2:p:95-127. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: http://www.hufs.ac.kr/user/hufsenglish/re_1.jsp .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.