IDEAS home Printed from https://ideas.repec.org/a/spr/comaot/v29y2023i1d10.1007_s10588-021-09353-w.html
   My bibliography  Save this article

Searching for explanations: testing social scientific methods in synthetic ground-truthed worlds

Author

Listed:
  • Aurora C. Schmidt

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Christopher J. Cameron

    (Cornell University)

  • Corey Lowman

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Joshua Brulé

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Amruta J. Deshpande

    (Graphika, Inc.)

  • Seyyed A. Fatemi

    (University of Hawaii)

  • Vladimir Barash

    (Graphika, Inc.)

  • Ariel M. Greenberg

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Cash J. Costello

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Eli S. Sherman

    (Johns Hopkins University)

  • Rohit Bhattacharya

    (Johns Hopkins University)

  • Liz McQuillan

    (Graphika, Inc.)

  • Alexander Perrone

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Yanni A. Kouskoulas

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • Clay Fink

    (Johns Hopkins University Applied Physics Laboratory (JHU/APL))

  • June Zhang

    (University of Hawaii)

  • Ilya Shpitser

    (Johns Hopkins University)

  • Michael W. Macy

    (Cornell University)

Abstract

A scientific model’s usefulness relies on its ability to explain phenomena, predict how such phenomena will be impacted by future interventions, and prescribe actions to achieve desired outcomes. We study methods for learning causal models that explain the behaviors of simulated “human” populations. Through the Ground Truth project, we solved a series of Challenges where our explanations, predictions and prescriptions were scored against ground truth information. We describe the processes that emerged for applying causal discovery, network analysis, agent-based modeling and other analytical methods to inform solutions to Challenge tasks. We present our team’s overall performance results on these Challenges and discuss implications for future efforts to validate social scientific research using simulation-based challenges.

Suggested Citation

  • Aurora C. Schmidt & Christopher J. Cameron & Corey Lowman & Joshua Brulé & Amruta J. Deshpande & Seyyed A. Fatemi & Vladimir Barash & Ariel M. Greenberg & Cash J. Costello & Eli S. Sherman & Rohit Bha, 2023. "Searching for explanations: testing social scientific methods in synthetic ground-truthed worlds," Computational and Mathematical Organization Theory, Springer, vol. 29(1), pages 156-187, March.
    • Handle: RePEc:spr:comaot:v:29:y:2023:i:1:d:10.1007_s10588-021-09353-w
    DOI: 10.1007/s10588-021-09353-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10588-021-09353-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10588-021-09353-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Ahmed Laatabi & Nicolas Marilleau & Tri Nguyen-Huu & Hassan Hbid & Mohamed Ait Babram, 2018. "ODD+2D: An ODD Based Protocol for Mapping Data to Empirical ABMs," Journal of Artificial Societies and Social Simulation, Journal of Artificial Societies and Social Simulation, vol. 21(2), pages 1-9.
    2. Tikka, Santtu & Karvanen, Juha, 2017. "Identifying Causal Effects with the R Package causaleffect," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 76(i12).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Raputsoane, Leroi, 2018. "Leader followership in monetary policy coordination," MPRA Paper 121905, University Library of Munich, Germany.
    2. Anshuka Anshuka & Floris F. Ogtrop & David Sanderson & Simone Z. Leao, 2022. "A systematic review of agent-based model for flood risk management and assessment using the ODD protocol," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 112(3), pages 2739-2771, July.
    3. Lixin Zhou & Jie Lin & Yanfeng Li & Zhenyu Zhang, 2020. "Innovation Diffusion of Mobile Applications in Social Networks: A Multi-Agent System," Sustainability, MDPI, vol. 12(7), pages 1-17, April.
    4. Jouni Helske & Santtu Tikka & Juha Karvanen, 2021. "Estimation of causal effects with small data in the presence of trapdoor variables," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(3), pages 1030-1051, July.
    5. Zanini, Sara, 2023. "Water challenges in socio-ecological systems: is human decision-making accounted for in the analysis of climate change adaptation options?," FEEM Working Papers 333364, Fondazione Eni Enrico Mattei (FEEM).
    6. Giacomo Ravaioli & Tiago Domingos & Ricardo F. M. Teixeira, 2023. "A Framework for Data-Driven Agent-Based Modelling of Agricultural Land Use," Land, MDPI, vol. 12(4), pages 1-17, March.
    7. Bauermann, Tom & Roos, Michael W. M. & Schaff, Frederik, 2020. "POSA: Policy implementation sensitivity analysis," Ruhr Economic Papers 854, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    8. Lauri Valkonen & Jouni Helske & Juha Karvanen, 2023. "Estimating the causal effect of timing on the reach of social media posts," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 32(2), pages 493-507, June.
    9. Christopher Hagedorn & Johannes Huegle & Rainer Schlosser, 2022. "Understanding unforeseen production downtimes in manufacturing processes using log data-driven causal reasoning," Journal of Intelligent Manufacturing, Springer, vol. 33(7), pages 2027-2043, October.
    10. Planque, Benjamin & Aarflot, Johanna M. & Buttay, Lucie & Carroll, JoLynn & Fransner, Filippa & Hansen, Cecilie & Husson, Bérengère & Langangen, Øystein & Lindstrøm, Ulf & Pedersen, Torstein & Primice, 2022. "A standard protocol for describing the evaluation of ecological models," Ecological Modelling, Elsevier, vol. 471(C).
    11. Raputsoane, Leroi, 2018. "Monetary policy coordination leader followership," MPRA Paper 85684, University Library of Munich, Germany.
    12. Brathwaite, Timothy & Walker, Joan L., 2018. "Causal inference in travel demand modeling (and the lack thereof)," Journal of choice modelling, Elsevier, vol. 26(C), pages 1-18.
    13. Sara Floriani Zanini, 2023. "Water challenges in socio-ecological systems: is human decision-making accounted for in the analysis of climate change adaptation options?," Working Papers 2023.06, Fondazione Eni Enrico Mattei.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:comaot:v:29:y:2023:i:1:d:10.1007_s10588-021-09353-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.