IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0069958.html
   My bibliography  Save this article

Comparing the Quality of Crowdsourced Data Contributed by Expert and Non-Experts

Author

Listed:
  • Linda See
  • Alexis Comber
  • Carl Salk
  • Steffen Fritz
  • Marijn van der Velde
  • Christoph Perger
  • Christian Schill
  • Ian McCallum
  • Florian Kraxner
  • Michael Obersteiner

Abstract

There is currently a lack of in-situ environmental data for the calibration and validation of remotely sensed products and for the development and verification of models. Crowdsourcing is increasingly being seen as one potentially powerful way of increasing the supply of in-situ data but there are a number of concerns over the subsequent use of the data, in particular over data quality. This paper examined crowdsourced data from the Geo-Wiki crowdsourcing tool for land cover validation to determine whether there were significant differences in quality between the answers provided by experts and non-experts in the domain of remote sensing and therefore the extent to which crowdsourced data describing human impact and land cover can be used in further scientific research. The results showed that there was little difference between experts and non-experts in identifying human impact although results varied by land cover while experts were better than non-experts in identifying the land cover type. This suggests the need to create training materials with more examples in those areas where difficulties in identification were encountered, and to offer some method for contributors to reflect on the information they contribute, perhaps by feeding back the evaluations of their contributed data or by making additional training materials available. Accuracies were also found to be higher when the volunteers were more consistent in their responses at a given location and when they indicated higher confidence, which suggests that these additional pieces of information could be used in the development of robust measures of quality in the future.

Suggested Citation

  • Linda See & Alexis Comber & Carl Salk & Steffen Fritz & Marijn van der Velde & Christoph Perger & Christian Schill & Ian McCallum & Florian Kraxner & Michael Obersteiner, 2013. "Comparing the Quality of Crowdsourced Data Contributed by Expert and Non-Experts," PLOS ONE, Public Library of Science, vol. 8(7), pages 1-11, July.
    • Handle: RePEc:plo:pone00:0069958
    DOI: 10.1371/journal.pone.0069958
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0069958
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0069958&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0069958?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jeremy Ginsberg & Matthew H. Mohebbi & Rajan S. Patel & Lynnette Brammer & Mark S. Smolinski & Larry Brilliant, 2009. "Detecting influenza epidemics using search engine query data," Nature, Nature, vol. 457(7232), pages 1012-1014, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Paul D. Juarez & Patricia Matthews-Juarez & Darryl B. Hood & Wansoo Im & Robert S. Levine & Barbara J. Kilbourne & Michael A. Langston & Mohammad Z. Al-Hamdan & William L. Crosson & Maurice G. Estes &, 2014. "The Public Health Exposome: A Population-Based, Exposure Science Approach to Health Disparities Research," IJERPH, MDPI, vol. 11(12), pages 1-30, December.
    2. Hone-Jay Chu & Yi-Chin Chen, 2018. "Crowdsourcing photograph locations for debris flow hot spot mapping," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 90(3), pages 1259-1276, February.
    3. Abolghasem Sadeghi-Niaraki & Mohammadreza Jelokhani-Niaraki & Soo-Mi Choi, 2020. "A Volunteered Geographic Information-Based Environmental Decision Support System for Waste Management and Decision Making," Sustainability, MDPI, vol. 12(15), pages 1-21, July.
    4. Andreas Spitz & Emőke-Ágnes Horvát, 2014. "Measuring Long-Term Impact Based on Network Centrality: Unraveling Cinematic Citations," PLOS ONE, Public Library of Science, vol. 9(10), pages 1-12, October.
    5. Frajer, Jindřich & Fiedor, David, 2021. "A historical curiosity or a source of accurate spatial information on historical land use? The issue of accuracy of old cadastres in the example of Josephian Cadastre from the Habsburg Empire," Land Use Policy, Elsevier, vol. 100(C).
    6. Barbosu, Sandra & Gans, Joshua S., 2022. "Storm crowds: Evidence from Zooniverse on crowd contribution design," Research Policy, Elsevier, vol. 51(1).
    7. Itai Kloog & Lara Ifat Kaufman & Kees De Hoogh, 2018. "Using Open Street Map Data in Environmental Exposure Assessment Studies: Eastern Massachusetts, Bern Region, and South Israel as a Case Study," IJERPH, MDPI, vol. 15(11), pages 1-21, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David H Chae & Sean Clouston & Mark L Hatzenbuehler & Michael R Kramer & Hannah L F Cooper & Sacoby M Wilson & Seth I Stephens-Davidowitz & Robert S Gold & Bruce G Link, 2015. "Association between an Internet-Based Measure of Area Racism and Black Mortality," PLOS ONE, Public Library of Science, vol. 10(4), pages 1-12, April.
    2. Xiaoli Wang & Shuangsheng Wu & C Raina MacIntyre & Hongbin Zhang & Weixian Shi & Xiaomin Peng & Wei Duan & Peng Yang & Yi Zhang & Quanyi Wang, 2015. "Using an Adjusted Serfling Regression Model to Improve the Early Warning at the Arrival of Peak Timing of Influenza in Beijing," PLOS ONE, Public Library of Science, vol. 10(3), pages 1-14, March.
    3. Ishani Chaudhuri & Parthajit Kayal, 2022. "Predicting Power of Ticker Search Volume in Indian Stock Market," Working Papers 2022-214, Madras School of Economics,Chennai,India.
    4. Yang, Xin & Pan, Bing & Evans, James A. & Lv, Benfu, 2015. "Forecasting Chinese tourist volume with search engine data," Tourism Management, Elsevier, vol. 46(C), pages 386-397.
    5. Kuchler, Theresa & Russel, Dominic & Stroebel, Johannes, 2022. "JUE Insight: The geographic spread of COVID-19 correlates with the structure of social networks as measured by Facebook," Journal of Urban Economics, Elsevier, vol. 127(C).
    6. Markowitz, Sara & Nesson, Erik & Robinson, Joshua J., 2019. "The effects of employment on influenza rates," Economics & Human Biology, Elsevier, vol. 34(C), pages 286-295.
    7. Bentzen, Jeanet Sinding, 2021. "In crisis, we pray: Religiosity and the COVID-19 pandemic," Journal of Economic Behavior & Organization, Elsevier, vol. 192(C), pages 541-583.
    8. Jesse T. Richman & Ryan J. Roberts, 2023. "Assessing Spurious Correlations in Big Search Data," Forecasting, MDPI, vol. 5(1), pages 1-12, February.
    9. Linus Schiöler & Marianne Fris�n, 2012. "Multivariate outbreak detection," Journal of Applied Statistics, Taylor & Francis Journals, vol. 39(2), pages 223-242, April.
    10. Sasikiran Kandula & Jeffrey Shaman, 2019. "Reappraising the utility of Google Flu Trends," PLOS Computational Biology, Public Library of Science, vol. 15(8), pages 1-16, August.
    11. Daniel E. O'Leary, 2024. "Toward an extended framework of exhaust data for predictive analytics: An empirical approach," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 31(2), June.
    12. Yangkun Huang & Xiaoping Xu & Sini Su, 2021. "Diverging from News Media: An Exploratory Study on the Changing Dynamics between Media and Public Attention on Cancer in China from 2011–2020," IJERPH, MDPI, vol. 18(16), pages 1-13, August.
    13. Vosen, Simeon & Schmidt, Torsten, 2012. "A monthly consumption indicator for Germany based on Internet search query data," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 19(7), pages 683-687.
    14. Klaus Ackermann & Simon D Angus & Paul A Raschky, 2017. "The Internet as Quantitative Social Science Platform: Insights from a Trillion Observations," Papers 1701.05632, arXiv.org.
    15. Edward L. Glaeser & Scott Duke Kominers & Michael Luca & Nikhil Naik, 2018. "Big Data And Big Cities: The Promises And Limitations Of Improved Measures Of Urban Life," Economic Inquiry, Western Economic Association International, vol. 56(1), pages 114-137, January.
    16. Sean Coogan & Zhixian Sui & David Raubenheimer, 2018. "Gluttony and guilt: monthly trends in internet search query data are comparable with national-level energy intake and dieting behavior," Palgrave Communications, Palgrave Macmillan, vol. 4(1), pages 1-9, December.
    17. Tobias Preis & Federico Botta & Helen Susannah Moat, 2020. "Sensing global tourism numbers with millions of publicly shared online photographs," Environment and Planning A, , vol. 52(3), pages 471-477, May.
    18. D'Amuri, Francesco & Marcucci, Juri, 2009. "‘Google it!’ Forecasting the US unemployment rate with a Google job search index," ISER Working Paper Series 2009-32, Institute for Social and Economic Research.
    19. Liwen Ling & Dabin Zhang & Shanying Chen & Amin W. Mugera, 2020. "Can online search data improve the forecast accuracy of pork price in China?," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(4), pages 671-686, July.
    20. Klaus Ackermann & Simon D Angus & Paul A Raschky, 2020. "Estimating Sleep and Work Hours from Alternative Data by Segmented Functional Classification Analysis, SFCA," SoDa Laboratories Working Paper Series 2020-04, Monash University, SoDa Laboratories.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0069958. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.