IDEAS home Printed from https://ideas.repec.org/p/ces/ceswps/_11275.html
   My bibliography  Save this paper

Experiments on Social Media

Author

Listed:
  • Guy Aridor
  • Rafael Jiménez-Durán
  • Ro'ee Levy
  • Lena Song

Abstract

We provide a practical guide to designing, conducting, and analyzing experiments using social media platforms. First, we discuss the benefits and challenges of using the targeting capabilities of advertisements on social media to recruit participants for a large class of experiments. Next, we outline the different types of interventions and their advantages and disadvantages. Finally, we summarize available compliance and outcome data, as well as the main limitations and challenges involved in the design and analysis of social media experiments. Throughout, we provide technical details that are helpful when implementing these experiments. Overall, we argue that experiments on social media are powerful not only for studying economic issues around social media and online platforms but also for experiments studying economic behavior more broadly.

Suggested Citation

  • Guy Aridor & Rafael Jiménez-Durán & Ro'ee Levy & Lena Song, 2024. "Experiments on Social Media," CESifo Working Paper Series 11275, CESifo.
    • Handle: RePEc:ces:ceswps:_11275
    as

    Download full text from publisher

    File URL: https://www.cesifo.org/DocDL/cesifo1_wp11275.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Bruno Crépon & Esther Duflo & Marc Gurgand & Roland Rathelot & Philippe Zamora, 2013. "Do Labor Market Policies have Displacement Effects? Evidence from a Clustered Randomized Experiment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 128(2), pages 531-580.
    2. Brett R. Gordon & Robert Moakler & Florian Zettelmeyer, 2023. "Close Enough? A Large-Scale Exploration of Non-Experimental Approaches to Advertising Measurement," Marketing Science, INFORMS, vol. 42(4), pages 768-793, July.
    3. Duflo, Esther & Glennerster, Rachel & Kremer, Michael, 2008. "Using Randomization in Development Economics Research: A Toolkit," Handbook of Development Economics, in: T. Paul Schultz & John A. Strauss (ed.), Handbook of Development Economics, edition 1, volume 4, chapter 61, pages 3895-3962, Elsevier.
    4. Justin Holz & Rafael Jiménez-Durán & Eduardo Laguna-Müggenburg, 2024. "Estimating the Distaste for Price Gouging with Incentivized Consumer Reports," American Economic Journal: Applied Economics, American Economic Association, vol. 16(1), pages 33-59, January.
    5. Luca Braghieri & Ro'ee Levy & Alexey Makarin, 2022. "Social Media and Mental Health," American Economic Review, American Economic Association, vol. 112(11), pages 3660-3693, November.
    6. Carattini, Stefano & Gillingham, Kenneth & Meng, Xiangyu & Yoeli, Erez, 2024. "Peer-to-peer solar and social rewards: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 219(C), pages 340-370.
    7. Emeric Henry & Ekaterina Zhuravskaya & Sergei Guriev, 2022. "Checking and Sharing Alt-Facts," American Economic Journal: Economic Policy, American Economic Association, vol. 14(3), pages 55-86, August.
    8. José Ramón Enríquez & Horacio Larreguy & John Marshall & Alberto Simpser, 2024. "Mass Political Information on Social Media: Facebook Ads, Electorate Saturation, and Electoral Accountability in Mexico," Journal of the European Economic Association, European Economic Association, vol. 22(4), pages 1678-1722.
    9. John A. List, 2020. "Non est Disputandum de Generalizability? A Glimpse into The External Validity Trial," NBER Working Papers 27535, National Bureau of Economic Research, Inc.
    10. Tesary Lin & Avner Strulov-Shlain, 2023. "Choice Architecture, Privacy Valuations, and Selection Bias in Consumer Data," Papers 2308.13496, arXiv.org.
    11. Leonardo Bursztyn & Benjamin R. Handel & Rafael Jimenez & Christopher Roth, 2023. "When Product Markets Become Collective Traps: The Case of Social Media," NBER Working Papers 31771, National Bureau of Economic Research, Inc.
    12. Amanda Y. Agan & Diag Davenport & Jens Ludwig & Sendhil Mullainathan, 2023. "Automating Automaticity: How the Context of Human Choice Affects the Extent of Algorithmic Bias," NBER Working Papers 30981, National Bureau of Economic Research, Inc.
    13. Hoong, Ruru, 2021. "Self control and smartphone use: An experimental study of soft commitment devices," European Economic Review, Elsevier, vol. 140(C).
    14. Ro'ee Levy, 2021. "Social Media, News Consumption, and Polarization: Evidence from a Field Experiment," American Economic Review, American Economic Association, vol. 111(3), pages 831-870, March.
    15. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    16. Emeric Henry & Ekaterina Zhuravskaya & Sergei Guriev, 2022. "Checking and Sharing Alt-Facts," PSE-Ecole d'économie de Paris (Postprint) halshs-03342759, HAL.
    17. David G. Rand & Alexander Peysakhovich & Gordon T. Kraft-Todd & George E. Newman & Owen Wurzbacher & Martin A. Nowak & Joshua D. Greene, 2014. "Social heuristics shape intuitive cooperation," Nature Communications, Nature, vol. 5(1), pages 1-12, May.
    18. Chiara Farronato & Andrey Fradkin & Chris Karr, 2024. "Webmunk: A New Tool for Studying Online Behavior and Digital Platforms," NBER Working Papers 32694, National Bureau of Economic Research, Inc.
    19. Leonardo Bursztyn, 2023. "When Product Markets Become Collective Traps: The Case of Social Media," ECONtribute Discussion Papers Series 260, University of Bonn and University of Cologne, Germany.
    20. Zhang, Baobao & Mildenberger, Matto & Howe, Peter D. & Marlon, Jennifer & Rosenthal, Seth A. & Leiserowitz, Anthony, 2020. "Quota sampling using Facebook advertisements," Political Science Research and Methods, Cambridge University Press, vol. 8(3), pages 558-564, July.
    21. Emeric Henry & Ekaterina Zhuravskaya & Sergei Guriev, 2022. "Checking and Sharing Alt-Facts," Post-Print halshs-03342759, HAL.
    22. Beknazar-Yuzbashev, George & Stalinski, Mateusz, 2022. "Do social media ads matter for political behavior? A field experiment," Journal of Public Economics, Elsevier, vol. 214(C).
    23. David Samuels & Cesar Zucco, 2014. "The Power of Partisanship in Brazil: Evidence from Survey Experiments," American Journal of Political Science, John Wiley & Sons, vol. 58(1), pages 212-225, January.
    24. Robert M. Bond & Christopher J. Fariss & Jason J. Jones & Adam D. I. Kramer & Cameron Marlow & Jaime E. Settle & James H. Fowler, 2012. "A 61-million-person experiment in social influence and political mobilization," Nature, Nature, vol. 489(7415), pages 295-298, September.
    25. Randall A. Lewis & Justin M. Rao, 2015. "The Unfavorable Economics of Measuring the Returns to Advertising," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 130(4), pages 1941-1973.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Aridor, Guy & Jiménez-Durán, Rafael & Levy, Ro'ee & Song, Lena, 2024. "The Economics of Social Media," CEPR Discussion Papers 18821, C.E.P.R. Discussion Papers.
    2. Jiménez Durán, Rafael & Muller, Karsten & Schwarz, Carlo, 2024. "The Effect of Content Moderation on Online and Offline Hate: Evidence from Germany’s NetzDG," CAGE Online Working Paper Series 701, Competitive Advantage in the Global Economy (CAGE).
    3. Ruben Enikolopov & Maria Petrova & Gianluca Russo & David Yanagizawa-Drott, 2024. "Socializing Alone: How Online Homophily Has Underminded Social Cohesion in the US," CESifo Working Paper Series 11375, CESifo.
    4. Ali, Ayesha & Qazi, Ihsan Ayyub, 2023. "Countering misinformation on social media through educational interventions: Evidence from a randomized experiment in Pakistan," Journal of Development Economics, Elsevier, vol. 163(C).
    5. repec:irs:cepswp:2024-01 is not listed on IDEAS
    6. Cariolle, Joël & Elkhateeb, Yasmine & Maurel, Mathilde, 2024. "Misinformation technology: Internet use and political misperceptions in Africa," Journal of Comparative Economics, Elsevier, vol. 52(2), pages 400-433.
    7. Bellodi, Luca & Docquier, Frédéric & Iandolo, Stefano & Morelli, Massimo & Turati, Riccardo, 2024. "Digging up Trenches: Populism, Selective Mobility, and the Political Polarization of Italian Municipalities," IZA Discussion Papers 16732, Institute of Labor Economics (IZA).
    8. Thomas Renault & David Restrepo Amariles & Aurore Troussel, 2024. "Collaboratively adding context to social media posts reduces the sharing of false news," Papers 2404.02803, arXiv.org.
    9. Beeder, Monica & Sørensen, Erik Ø., 2023. "Replication Report: Checking and Sharing Alt-Facts," I4R Discussion Paper Series 34, The Institute for Replication (I4R).
    10. Andrea Mattozzi & Samuel Nocito & Francesco Sobbrio, 2022. "Fact-Checking Politicians," CESifo Working Paper Series 10122, CESifo.
    11. Ek, Claes & Samahita, Margaret, 2023. "Too much commitment? An online experiment with tempting YouTube content," Journal of Economic Behavior & Organization, Elsevier, vol. 208(C), pages 21-38.
    12. Dana Sisak & Philipp Denter, 2024. "Information Sharing with Social Image Concerns and the Spread of Fake News," Papers 2410.19557, arXiv.org, revised Oct 2024.
    13. Choi, Syngjoo & Choi, Chung-Yoon & Kim, Seonghoon, 2023. "Tackling misperceptions about immigrants with fact-checking interventions: A randomized survey experiment," Labour Economics, Elsevier, vol. 84(C).
    14. Guy Aridor, 2022. "Measuring Substitution Patterns in the Attention Economy: An Experimental Approach," CESifo Working Paper Series 10190, CESifo.
    15. Kazi Iqbal & Asad Islam & John List & Vy Nguyen, 2021. "Myopic Loss Aversion and Investment Decisions: From the Laboratory to the Field," Framed Field Experiments 000730, The Field Experiments Website.
    16. Carattini, Stefano & Gillingham, Kenneth & Meng, Xiangyu & Yoeli, Erez, 2024. "Peer-to-peer solar and social rewards: Evidence from a field experiment," Journal of Economic Behavior & Organization, Elsevier, vol. 219(C), pages 340-370.
    17. Ingar K. Haaland & Christopher Roth & Stefanie Stantcheva & Johannes Wohlfart, 2024. "Measuring What is Top of Mind," NBER Working Papers 32421, National Bureau of Economic Research, Inc.
    18. Jörg Peters & Jörg Langbein & Gareth Roberts, 2018. "Generalization in the Tropics – Development Policy, Randomized Controlled Trials, and External Validity," The World Bank Research Observer, World Bank, vol. 33(1), pages 34-64.
    19. List, John A. & Momeni, Fatemeh & Zenou, Yves, 2020. "The Social Side of Early Human Capital Formation: Using a Field Experiment to Estimate the Causal Impact of Neighborhoods," IZA Discussion Papers 13966, Institute of Labor Economics (IZA).
    20. Giuberti Coutinho, Lorena, 2021. "Political polarization and the impact of internet and social media use in Brazil," MERIT Working Papers 2021-032, United Nations University - Maastricht Economic and Social Research Institute on Innovation and Technology (MERIT).
    21. Chopra, Felix & Haaland, Ingar & Roth, Christopher, 2022. "Do people demand fact-checked news? Evidence from U.S. Democrats," Journal of Public Economics, Elsevier, vol. 205(C).

    More about this item

    Keywords

    social media; experiments; digital interventions; subject recruitment; experiment design;
    All these keywords.

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • L82 - Industrial Organization - - Industry Studies: Services - - - Entertainment; Media
    • L96 - Industrial Organization - - Industry Studies: Transportation and Utilities - - - Telecommunications

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ces:ceswps:_11275. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Klaus Wohlrabe (email available below). General contact details of provider: https://edirc.repec.org/data/cesifde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.