IDEAS home Printed from https://ideas.repec.org/p/bdp/dpaper/0055.html
   My bibliography  Save this paper

E-Learning at Universities: Does Starting with Difficult Questions Affect Student Performance?

Author

Listed:
  • Agata Galkiewicz
  • Jan Marcus
  • Thomas Siedler

Abstract

To reduce cheating in written tests and exams, assessors often randomly vary the order of questions across students. However, little is known about the potential unintended side effects of question order. This paper examines whether randomizing students to start with an easier or harder question makes a difference to overall assessment performance in incentivized testing situations under time pressure. Using data from more than 8,000 online tests and exams administered in econometrics and statistics courses at two of Germany's largest universities, we find no evidence that the difficulty of the first question(s) has an effect on overall assessment performance. Our findings are good news for people designing (online) assessments, because randomizing the order of questions can be used as an effective tool to mitigate cheating, but does not affect students' overall performance.

Suggested Citation

  • Agata Galkiewicz & Jan Marcus & Thomas Siedler, 2024. "E-Learning at Universities: Does Starting with Difficult Questions Affect Student Performance?," Berlin School of Economics Discussion Papers 0055, Berlin School of Economics.
    • Handle: RePEc:bdp:dpaper:0055
    DOI: 10.48462/opus4-5657
    as

    Download full text from publisher

    File URL: https://opus4.kobv.de/opus4-hsog/files/5657/BSoE_DP_0055.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.48462/opus4-5657?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Feld, Jan & Salamanca, Nicolás & Zölitz, Ulf, 2019. "Students are almost as effective as professors in university teaching," Economics of Education Review, Elsevier, vol. 73(C).
    2. Oskar R. Harmon & James Lambrinos, 2008. "Are Online Exams an Invitation to Cheat?," The Journal of Economic Education, Taylor & Francis Journals, vol. 39(2), pages 116-125, April.
    3. Claudio Lucifora & Marco Tonello, 2020. "Monitoring and Sanctioning Cheating at School: What Works? Evidence from a National Evaluation Program," Journal of Human Capital, University of Chicago Press, vol. 14(4), pages 584-616.
    4. Grace Eau & Derek Hoodin & Tareena Musaddiq, 2022. "Testing the effects of adaptive learning courseware on student performance: An experimental approach," Southern Economic Journal, John Wiley & Sons, vol. 88(3), pages 1086-1118, January.
    5. Caroline M. Hoxby, 2014. "The Economics of Online Postsecondary Education: MOOCs, Nonselective Education, and Highly Selective Education," American Economic Review, American Economic Association, vol. 104(5), pages 528-533, May.
    6. De Paola, Maria & Gioia, Francesca & Scoppa, Vincenzo, 2023. "Online teaching, procrastination and student achievement," Economics of Education Review, Elsevier, vol. 94(C).
    7. Bilen, Eren & Matros, Alexander, 2021. "Online cheating amid COVID-19," Journal of Economic Behavior & Organization, Elsevier, vol. 182(C), pages 196-211.
    8. Hill, Andrew J. & LoPalo, Melissa, 2024. "The effects of online vs in-class testing in moderate-stakes college environments," Economics of Education Review, Elsevier, vol. 98(C).
    9. Feld, Jan & Salamanca, Nicolás & Zölitz, Ulf, 2019. "Students are almost as effective as professors in university teaching," Economics of Education Review, Elsevier, vol. 73(C).
    10. Byron W. Brown & Carl E. Liedholm, 2002. "Can Web Courses Replace the Classroom in Principles of Microeconomics?," American Economic Review, American Economic Association, vol. 92(2), pages 444-448, May.
    11. Aaron Swoboda & Lauren Feiler, 2016. "Measuring the Effect of Blended Learning: Evidence from a Selective Liberal Arts College," American Economic Review, American Economic Association, vol. 106(5), pages 368-372, May.
    12. Eric P. Bettinger & Lindsay Fox & Susanna Loeb & Eric S. Taylor, 2017. "Virtual Classrooms: How Online College Courses Affect Student Success," American Economic Review, American Economic Association, vol. 107(9), pages 2855-2875, September.
    13. David J. Deming & Claudia Goldin & Lawrence F. Katz & Noam Yuchtman, 2015. "Can Online Learning Bend the Higher Education Cost Curve?," American Economic Review, American Economic Association, vol. 105(5), pages 496-501, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hoque, Nazmul & Basher, Syed Abul & A.K. Enamul, Haque, 2022. "Do Students Perform Better in Online Delivery of Education? Evidence from Bangladesh," MPRA Paper 112981, University Library of Munich, Germany.
    2. M Paula Cacault & Christian Hildebrand & Jérémy Laurent-Lucchetti & Michele Pellizzari, 2021. "Distance Learning in Higher Education: Evidence from a Randomized Experiment [A Randomized Assessment of Online Learning]," Journal of the European Economic Association, European Economic Association, vol. 19(4), pages 2322-2372.
    3. Michael S. Kofoed & Lucas Gebhart & Dallas Gilmore & Ryan Moschitto, 2024. "Zooming to Class? Experimental Evidence on College Students' Online Learning during COVID-19," American Economic Review: Insights, American Economic Association, vol. 6(3), pages 324-340, September.
    4. Merkus, Erik & Schafmeister, Felix, 2021. "The role of in-person tutorials in higher education," Economics Letters, Elsevier, vol. 201(C).
    5. Hardt, David & Nagler, Markus & Rincke, Johannes, 2022. "Can peer mentoring improve online teaching effectiveness? An RCT during the COVID-19 pandemic," Labour Economics, Elsevier, vol. 78(C).
    6. Benjamin Faber & Rosa Sanchis-Guarner & Felix Weinhardt, 2015. "ICT and Education: Evidence from Student Home Addresses," SERC Discussion Papers 0186, Centre for Economic Performance, LSE.
    7. Benjamin T. Skinner, 2019. "Making the Connection: Broadband Access and Online Course Enrollment at Public Open Admissions Institutions," Research in Higher Education, Springer;Association for Institutional Research, vol. 60(7), pages 960-999, November.
    8. Caroline M. Hoxby, 2018. "Online Postsecondary Education and the Higher Education Tax Benefits: An Analysis with Implications for Tax Administration," NBER Chapters, in: Tax Policy and the Economy, Volume 32, pages 45-106, National Bureau of Economic Research, Inc.
    9. Eric P. Chiang & Jose J. Vazquez, 2018. "Using Technology to Complete the Natural Learning Path in a Principles of Economics Course," Journal of Economics Teaching, Journal of Economics Teaching, vol. 2(2), pages 104-114, January.
    10. Marigee Bacolod & Stephen Mehay & Elda Pema, 2018. "Who succeeds in distance learning? Evidence from quantile panel data estimation," Southern Economic Journal, John Wiley & Sons, vol. 84(4), pages 1129-1145, April.
    11. Eric P. Bettinger & Lindsay Fox & Susanna Loeb & Eric S. Taylor, 2017. "Virtual Classrooms: How Online College Courses Affect Student Success," American Economic Review, American Economic Association, vol. 107(9), pages 2855-2875, September.
    12. Flip Klijn & Mehdi Mdaghri Alaoui & Marc Vorsatz, 2024. "Online Academic Exams: Does Multiplicity of Exam Versions Mitigate Cheating?," Working Papers 1430, Barcelona School of Economics.
    13. Cheryl J. Wachenheim, 2009. "Final Exam Scores in Introductory Economics Courses: Effect of Course Delivery Method and Proctoring," Review of Agricultural Economics, Agricultural and Applied Economics Association, vol. 31(3), pages 640-652.
    14. Lisa Barrow & Wesley T. Morris & Lauren Sartain, 2024. "The Expanding Landscape of Online Education: Who Engages and How They Fare," Journal of Labor Economics, University of Chicago Press, vol. 42(S1), pages 417-443.
    15. Wuthisatian, Rattaphon, 2020. "Student exam performance in different proctored environments: Evidence from an online economics course," International Review of Economics Education, Elsevier, vol. 35(C).
    16. Insler, Michael & McQuoid, Alexander F. & Rahman, Ahmed S. & Smith, Katherine, 2021. "Fear and Loathing in the Classroom: Why Does Teacher Quality Matter?," IZA Discussion Papers 14036, Institute of Labor Economics (IZA).
    17. Justin C. Ortagus & Lijing Yang, 2018. "An Examination of the Influence of Decreases in State Appropriations on Online Enrollment at Public Universities," Research in Higher Education, Springer;Association for Institutional Research, vol. 59(7), pages 847-865, November.
    18. Jacqmin, Julien, 2019. "Providing MOOCs: A FUN way to enroll students?," Information Economics and Policy, Elsevier, vol. 48(C), pages 32-39.
    19. Griffith, Amanda L. & Main, Joyce B., 2021. "The role of the teaching assistant: Female role models in the classroom," Economics of Education Review, Elsevier, vol. 85(C).
    20. Hall, Caroline & Hardoy, Inés & Lundin, Martin, 2022. "Schooling in the Nordic countries during the COVID-19 pandemic," Working Paper Series 2022:13, IFAU - Institute for Evaluation of Labour Market and Education Policy.

    More about this item

    Keywords

    education; university students; question order; randomization; e-learning; teaching of economics;
    All these keywords.

    JEL classification:

    • A22 - General Economics and Teaching - - Economic Education and Teaching of Economics - - - Undergraduate
    • I23 - Health, Education, and Welfare - - Education - - - Higher Education; Research Institutions

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bdp:dpaper:0055. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christian Reiter (email available below). General contact details of provider: https://edirc.repec.org/data/bdpemde.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.