IDEAS home Printed from https://ideas.repec.org/a/wly/riskan/v30y2010i6p906-915.html
   My bibliography  Save this article

Comparison of Scoring Systems for Invasive Pests Using ROC Analysis and Monte Carlo Simulations

Author

Listed:
  • David Makowski
  • Murthy Narasimha Mittinty

Abstract

Different international plant protection organisations advocate different schemes for conducting pest risk assessments. Most of these schemes use structured questionnaire in which experts are asked to score several items using an ordinal scale. The scores are then combined using a range of procedures, such as simple arithmetic mean, weighted averages, multiplication of scores, and cumulative sums. The most useful schemes will correctly identify harmful pests and identify ones that are not. As the quality of a pest risk assessment can depend on the characteristics of the scoring system used by the risk assessors (i.e., on the number of points of the scale and on the method used for combining the component scores), it is important to assess and compare the performance of different scoring systems. In this article, we proposed a new method for assessing scoring systems. Its principle is to simulate virtual data using a stochastic model and, then, to estimate sensitivity and specificity values from these data for different scoring systems. The interest of our approach was illustrated in a case study where several scoring systems were compared. Data for this analysis were generated using a probabilistic model describing the pest introduction process. The generated data were then used to simulate the outcome of scoring systems and to assess the accuracy of the decisions about positive and negative introduction. The results showed that ordinal scales with at most 5 or 6 points were sufficient and that the multiplication‐based scoring systems performed better than their sum‐based counterparts. The proposed method could be used in the future to assess a great diversity of scoring systems.

Suggested Citation

  • David Makowski & Murthy Narasimha Mittinty, 2010. "Comparison of Scoring Systems for Invasive Pests Using ROC Analysis and Monte Carlo Simulations," Risk Analysis, John Wiley & Sons, vol. 30(6), pages 906-915, June.
  • Handle: RePEc:wly:riskan:v:30:y:2010:i:6:p:906-915
    DOI: 10.1111/j.1539-6924.2010.01393.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1539-6924.2010.01393.x
    Download Restriction: no

    File URL: https://libkey.io/10.1111/j.1539-6924.2010.01393.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gordon H. Copp & Lorenzo Vilizzi & John Mumford & Gemma V. Fenwick & Michael J. Godard & Rodolphe E. Gozlan, 2009. "Calibration of FISK, an Invasiveness Screening Tool for Nonnative Freshwater Fishes," Risk Analysis, John Wiley & Sons, vol. 29(3), pages 457-467, March.
    2. Hughes, G. & Madden, L. V., 2003. "Evaluating predictive models with application in regulatory policy for invasive weeds," Agricultural Systems, Elsevier, vol. 76(2), pages 755-774, May.
    3. Peterson, A. Townsend & Papeş, Monica & Soberón, Jorge, 2008. "Rethinking receiver operating characteristic analysis applications in ecological niche modeling," Ecological Modelling, Elsevier, vol. 213(1), pages 63-72.
    4. Trond Rafoss, 2003. "Spatial Stochastic Simulation Offers Potential as a Quantitative Method for Pest Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 23(4), pages 651-661, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mark Gibbs, 2011. "Ecological Risk Assessment, Prediction, and Assessing Risk Predictions," Risk Analysis, John Wiley & Sons, vol. 31(11), pages 1784-1788, November.
    2. Hong Yao & Xin Qian & Hong Yin & Hailong Gao & Yulei Wang, 2015. "Regional Risk Assessment for Point Source Pollution Based on a Water Quality Model of the Taipu River, China," Risk Analysis, John Wiley & Sons, vol. 35(2), pages 265-277, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Wiltshire, Kathryn H & Tanner, Jason E, 2020. "Comparing maximum entropy modelling methods to inform aquaculture site selection for novel seaweed species," Ecological Modelling, Elsevier, vol. 429(C).
    2. Frank H. Koch & Denys Yemshanov & Daniel W. McKenney & William D. Smith, 2009. "Evaluating Critical Uncertainty Thresholds in a Spatial Model of Forest Pest Invasion Risk," Risk Analysis, John Wiley & Sons, vol. 29(9), pages 1227-1241, September.
    3. Václavík, Tomáš & Meentemeyer, Ross K., 2009. "Invasive species distribution modeling (iSDM): Are absence data and dispersal constraints needed to predict actual distributions?," Ecological Modelling, Elsevier, vol. 220(23), pages 3248-3258.
    4. Wongsathit Wongloet & Prach Kongthong & Aingorn Chaiyes & Worapong Singchat & Warong Suksavate & Nattakan Ariyaraphong & Thitipong Panthum & Artem Lisachov & Kitipong Jaisamut & Jumaporn Sonongbua & T, 2023. "Genetic Monitoring of the Last Captive Population of Greater Mouse-Deer on the Thai Mainland and Prediction of Habitat Suitability before Reintroduction," Sustainability, MDPI, vol. 15(4), pages 1-22, February.
    5. Inês Silva & Matthew Crane & Pongthep Suwanwaree & Colin Strine & Matt Goode, 2018. "Using dynamic Brownian Bridge Movement Models to identify home range size and movement patterns in king cobras," PLOS ONE, Public Library of Science, vol. 13(9), pages 1-20, September.
    6. Wei Yang & Yuanxu Ma & Linhai Jing & Siyuan Wang & Zhongchang Sun & Yunwei Tang & Hui Li, 2022. "Differential Impacts of Climatic and Land Use Changes on Habitat Suitability and Protected Area Adequacy across the Asian Elephant’s Range," Sustainability, MDPI, vol. 14(9), pages 1-22, April.
    7. Ramos, Rodrigo Soares & Kumar, Lalit & Shabani, Farzin & Picanço, Marcelo Coutinho, 2019. "Risk of spread of tomato yellow leaf curl virus (TYLCV) in tomato crops under various climate change scenarios," Agricultural Systems, Elsevier, vol. 173(C), pages 524-535.
    8. Yemshanov, Denys & Haight, Robert G. & Koch, Frank H. & Lu, Bo & Venette, Robert & Fournier, Ronald E. & Turgeon, Jean J., 2017. "Robust Surveillance and Control of Invasive Species Using a Scenario Optimization Approach," Ecological Economics, Elsevier, vol. 133(C), pages 86-98.
    9. Liu, Canran & White, Matt & Newell, Graeme & Griffioen, Peter, 2013. "Species distribution modelling for conservation planning in Victoria, Australia," Ecological Modelling, Elsevier, vol. 249(C), pages 68-74.
    10. Denys Yemshanov & Frank H. Koch & Yakov Ben‐Haim & William D. Smith, 2010. "Robustness of Risk Maps and Survey Networks to Knowledge Gaps About a New Invasive Pest," Risk Analysis, John Wiley & Sons, vol. 30(2), pages 261-276, February.
    11. Lin, Yu-Pin & Wang, Cheng-Long & Yu, Hsiao-Hsuan & Huang, Chung-Wei & Wang, Yung-Chieh & Chen, Yu-Wen & Wu, Wei-Yao, 2011. "Monitoring and estimating the flow conditions and fish presence probability under various flow conditions at reach scale using genetic algorithms and kriging methods," Ecological Modelling, Elsevier, vol. 222(3), pages 762-775.
    12. Srivastava, Vivek & Carroll, Allan L., 2023. "Dynamic distribution modelling using a native invasive species, the mountain pine beetle," Ecological Modelling, Elsevier, vol. 482(C).
    13. Gisel Garza & Crystian Sadiel Venegas Barrera & Jon Dale & José Guadalupe Martínez-Ávalos & Teresa Patricia Feria Arroyo, 2022. "Towards Conserving Crop Wild Relatives along the Texas–Mexico Border: The Case of Manihot walkerae," Sustainability, MDPI, vol. 14(9), pages 1-15, April.
    14. Götz Schroth & Peter Läderach & Armando Isaac Martinez-Valle & Christian Bunn, 2017. "From site-level to regional adaptation planning for tropical commodities: cocoa in West Africa," Mitigation and Adaptation Strategies for Global Change, Springer, vol. 22(6), pages 903-927, August.
    15. Martín, Gerardo & Yáñez-Arenas, Carlos & Chiappa-Carrara, Xavier, 2022. "Discrepancies between point process models and environmental envelopes identify the niche centroid – geography configuration," Ecological Modelling, Elsevier, vol. 469(C).
    16. Soria-Auza, Rodrigo W. & Kessler, Michael & Bach, Kerstin & Barajas-Barbosa, Paola M. & Lehnert, Marcus & Herzog, Sebastian K. & Böhner, Jürgen, 2010. "Impact of the quality of climate models for modelling species occurrences in countries with poor climatic documentation: a case study from Bolivia," Ecological Modelling, Elsevier, vol. 221(8), pages 1221-1229.
    17. Yinglian Qi & Xiaoyan Pu & Yaxiong Li & Dingai Li & Mingrui Huang & Xuan Zheng & Jiaxin Guo & Zhi Chen, 2022. "Prediction of Suitable Distribution Area of Plateau pika ( Ochotona curzoniae ) in the Qinghai–Tibet Plateau under Shared Socioeconomic Pathways (SSPs)," Sustainability, MDPI, vol. 14(19), pages 1-23, September.
    18. Carlos Yañez-Arenas & A. Townsend Peterson & Karla Rodríguez-Medina & Narayani Barve, 2016. "Mapping current and future potential snakebite risk in the new world," Climatic Change, Springer, vol. 134(4), pages 697-711, February.
    19. Daniela Remolina-Figueroa & David A. Prieto-Torres & Wesley Dáttilo & Ernesto Salgado Díaz & Laura E. Nuñez Rosas & Claudia Rodríguez-Flores & Adolfo G. Navarro-Sigüenza & María del Coro Arizmendi, 2022. "Together forever? Hummingbird-plant relationships in the face of climate warming," Climatic Change, Springer, vol. 175(1), pages 1-21, November.
    20. Carlos Yañez-Arenas & A. Townsend Peterson & Karla Rodríguez-Medina & Narayani Barve, 2016. "Mapping current and future potential snakebite risk in the new world," Climatic Change, Springer, vol. 134(4), pages 697-711, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:riskan:v:30:y:2010:i:6:p:906-915. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1539-6924 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.