IDEAS home Printed from https://ideas.repec.org/a/taf/ginixx/v48y2022i4p860-896.html
   My bibliography  Save this article

United they stand: Findings from an escalation prediction competition

Author

Listed:
  • Paola Vesco
  • Håvard Hegre
  • Michael Colaresi
  • Remco Bastiaan Jansen
  • Adeline Lo
  • Gregor Reisch
  • Nils B. Weidmann

Abstract

This article presents results and lessons learned from a prediction competition organized by ViEWS to improve collective scientific knowledge on forecasting (de-)escalation in Africa. The competition call asked participants to forecast changes in state-based violence for the true future (October 2020–March 2021) as well as for a held-out test partition. An external scoring committee, independent from both the organizers and participants, was formed to evaluate the models based on both qualitative and quantitative criteria, including performance, novelty, uniqueness, and replicability. All models contributed to advance the research frontier by providing novel methodological or theoretical insight, including new data, or adopting innovative model specifications. While we discuss several facets of the competition that could be improved moving forward, the collection passes an important test. When we build a simple ensemble prediction model—which draws on the unique insights of each contribution to differing degrees—we can measure an improvement in the prediction from the group, over and above what the average individual model can achieve. This wisdom of the crowd effect suggests that future competitions that build on both the successes and failures of ours, can contribute to scientific knowledge by incentivizing diverse contributions as well as focusing a group’s attention on a common problem.Este artículo presenta los resultados y las enseñanzas extraídas en el marco de un certamen de predicción organizado por los responsables del proyecto Sistema de Alerta Temprana de Violencia (Violence Early-Warning System, ViEWS) con el propósito de mejorar los conocimientos científicos colectivos sobre la previsión de la (des)escalada en el continente africano. En el certamen se pidió a los participantes que desarrollaran una previsión con respecto a los cambios en la violencia estatal para el futuro real (de octubre de 2020 a marzo de 2021), así como para una muestra de prueba que se mantendría. Se formó un comité de calificación externo, independiente tanto de los organizadores como de los participantes, para evaluar los modelos en función de criterios cualitativos y cuantitativos, como el rendimiento, la novedad, la singularidad y la replicabilidad. Todos los modelos contribuyeron a avanzar en la frontera de la investigación mediante el aporte de nuevos conocimientos metodológicos o teóricos, la inclusión de nuevos datos o la adopción de especificaciones innovadoras del modelo. Aunque se debarió sobre varios aspectos del certamen que podrían mejorarse de cara al futuro, lo que se recopiló pasó una prueba importante. Cuando se construye un simple modelo de predicción de conjunto, que se basa en los conocimientos únicos de cada contribución en diferentes grados, se puede medir una mejora en la predicción del grupo, por encima de lo que el modelo individual promedio puede lograr. Este efecto de la sabiduría de la multitud sugiere que los futuros certámenes que se basen tanto en los éxitos como en los fracasos propios, pueden contribuir al conocimiento científico incentivando diversas contribuciones, así como centrando la atención de un grupo en un problema común.Cet article présente les résultats et les enseignements tirés d’un concours de prédiction organisé par ViEWS (Violence early-warning system, système d’alerte précoce sur la violence) pour améliorer nos connaissances scientifiques collectives en prévision de la (dés)escalade de la violence sur le continent africain. L’appel à concours demandait aux participants de prévoir les évolutions de la violence étatique pour le futur réel (octobre 2020-mars 2021) ainsi que pour une partition test retenue. Un comité de notation externe, indépendant à la fois des organisateurs et des participants, a été constitué pour évaluer les modèles à la fois sur des critères qualitatifs et quantitatifs, notamment sur leurs performances, leur nouveauté, leur unicité et leur reproductibilité. Tous les modèles ont contribué à faire avancer la frontière des recherches en apportant un éclairage méthodologique ou théorique inédit, en incluant de nouvelles données ou en adoptant des caractéristiques de modèle innovantes. Bien que nous abordions plusieurs facettes du concours qui pourraient être améliorées en allant de l’avant, l’ensemble de modèles a réussi un test important. Lorsque nous concevons un modèle de prédiction par ensemble simple - qui s’appuie sur les renseignements uniques de chaque contribution aux différents degrés -, nous pouvons mesurer une amélioration de la prédiction du groupe par rapport à ce que le modèle individuel moyen permet d’obtenir. Cet effet de sagesse de la foule suggère que les futurs concours, qui s’appuieront à la fois sur les réussites et les échecs du nôtre, pourront contribuer aux connaissances scientifiques en encourageant des contributions diverses et en concentrant l’attention d’un groupe sur un problème commun.

Suggested Citation

  • Paola Vesco & Håvard Hegre & Michael Colaresi & Remco Bastiaan Jansen & Adeline Lo & Gregor Reisch & Nils B. Weidmann, 2022. "United they stand: Findings from an escalation prediction competition," International Interactions, Taylor & Francis Journals, vol. 48(4), pages 860-896, July.
  • Handle: RePEc:taf:ginixx:v:48:y:2022:i:4:p:860-896
    DOI: 10.1080/03050629.2022.2029856
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/03050629.2022.2029856
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1080/03050629.2022.2029856?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mueller, H. & Rauh, C. & Seimon, B., 2024. "Introducing a Global Dataset on Conflict Forecasts and News Topics," Janeway Institute Working Papers 2402, Faculty of Economics, University of Cambridge.
    2. Racek, Daniel & Thurner, Paul & Kauermann, Goeran, 2024. "Integrating Spatio-temporal Diffusion into Statistical Forecasting Models of Armed Conflict via Non-parametric Smoothing," OSF Preprints q59dr, Center for Open Science.
    3. Rød, Espen Geelmuyden & Gåsste, Tim & Hegre, Håvard, 2024. "A review and comparison of conflict early warning systems," International Journal of Forecasting, Elsevier, vol. 40(1), pages 96-112.
    4. Racek, Daniel & Thurner, Paul W. & Davidson, Brittany I. & Zhu, Xiao Xiang & Kauermann, Göran, 2024. "Conflict forecasting using remote sensing data: An application to the Syrian civil war," International Journal of Forecasting, Elsevier, vol. 40(1), pages 373-391.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:ginixx:v:48:y:2022:i:4:p:860-896. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/GINI20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.