IDEAS home Printed from https://ideas.repec.org/a/eee/intell/v101y2023ics0160289623000740.html
   My bibliography  Save this article

Unsuccessful and successful complex problem solvers – A log file analysis of complex problem solving strategies across multiple tasks

Author

Listed:
  • Nicolay, Björn
  • Krieger, Florian
  • Kuhn, Jörg-Tobias
  • Graesser, Arthur C.
  • Ifenthaler, Dirk
  • Baker, Ryan
  • Greiff, Samuel

Abstract

Complex problem solving (CPS) is a key competence in educational contexts with strong conceptual links to students' overall intelligence. However, the mechanisms underlying successful CPS are not fully understood. Therefore, this study investigated several factors presumed to be relevant to CPS success using log file data to code each individual student action during six CPS tasks with different characteristics (N = 1276). We coded individual strategy combinations per student for each item for different strategy combinations of vary-one-thing-at-a-time (VOTAT), hold-one-thing-at-a-time (HOTAT), vary-no-thing-at-a-time (NOTAT), and change-all (CA). Results from generalized linear mixed models showed that CPS success was likely to be achieved by using VOTAT. However, there was an increased chance of solving an item when additional strategies, such as NOTAT or NOTAT plus HOTAT were used. This result was moderated by the presence/absence of eigendynamics as an important determinant of item difficulty. Strategy combinations of VOTAT together with other strategies (all including NOTAT) showed higher chances of CPS success when eigendynamics were present. Additionally, higher chances of solving an item when using VOTAT with additional strategies was demonstrated for items without eigendynamics. Overall, our results suggest that flexibility in strategy application is the driving force for successful CPS performance. Implications are discussed in light of the presumed benefit of pedagogically relevant metacognitive skills, such as planning, monitoring and reflecting, for CPS success. Based on our findings, we provide specific recommendations for the development of computer-based learning simulations to train CPS and related competencies, ultimately enhancing students' skills in educational contexts.

Suggested Citation

  • Nicolay, Björn & Krieger, Florian & Kuhn, Jörg-Tobias & Graesser, Arthur C. & Ifenthaler, Dirk & Baker, Ryan & Greiff, Samuel, 2023. "Unsuccessful and successful complex problem solvers – A log file analysis of complex problem solving strategies across multiple tasks," Intelligence, Elsevier, vol. 101(C).
    • Handle: RePEc:eee:intell:v:101:y:2023:i:c:s0160289623000740
    DOI: 10.1016/j.intell.2023.101793
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0160289623000740
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.intell.2023.101793?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Michela Gnaldi & Silvia Bacci & Thiemo Kunze & Samuel Greiff, 2020. "Students’ Complex Problem Solving Profiles," Psychometrika, Springer;The Psychometric Society, vol. 85(2), pages 469-501, June.
    2. Dellaert, Benedict G.C. & Arentze, Theo & Horeni, Oliver & Timmermans, Harry J.P., 2017. "Deriving attribute utilities from mental representations of complex decisions," Journal of choice modelling, Elsevier, vol. 22(C), pages 24-38.
    3. Reijnders, Laurie S.M. & de Vries, Gaaitzen J., 2018. "Technology, offshoring and the rise of non-routine jobs," Journal of Development Economics, Elsevier, vol. 135(C), pages 412-432.
    4. Stadler, Matthias & Niepel, Christoph & Greiff, Samuel, 2019. "Differentiating between static and complex problems: A theoretical framework and its empirical validation," Intelligence, Elsevier, vol. 72(C), pages 1-12.
    5. Luca Marcolin & Sébastien Miroudot & Mariagrazia Squicciarini, 2019. "To be (routine) or not to be (routine), that is the question: a cross-country task-based answer†," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 28(3), pages 477-501.
    6. Rudolph, Julia & Niepel, Christoph & Greiff, Samuel & Goldhammer, Frank & Kröner, Stephan, 2017. "Metacognitive confidence judgments and their link to complex problem solving," Intelligence, Elsevier, vol. 63(C), pages 1-8.
    7. Bates, Douglas & Mächler, Martin & Bolker, Ben & Walker, Steve, 2015. "Fitting Linear Mixed-Effects Models Using lme4," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 67(i01).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. de Vries, Gaaitzen J. & Gentile, Elisabetta & Miroudot, Sébastien & Wacker, Konstantin M., 2020. "The rise of robots and the fall of routine jobs," Labour Economics, Elsevier, vol. 66(C).
    2. JANSSENS, Jochen & DE CORTE, Annelies & SÖRENSEN, Kenneth, 2016. "Water distribution network design optimisation with respect to reliability," Working Papers 2016007, University of Antwerp, Faculty of Business and Economics.
    3. Raymond Hernandez & Elizabeth A. Pyatak & Cheryl L. P. Vigen & Haomiao Jin & Stefan Schneider & Donna Spruijt-Metz & Shawn C. Roll, 2021. "Understanding Worker Well-Being Relative to High-Workload and Recovery Activities across a Whole Day: Pilot Testing an Ecological Momentary Assessment Technique," IJERPH, MDPI, vol. 18(19), pages 1-17, October.
    4. Elisabeth Beckmann & Lukas Olbrich & Joseph Sakshaug, 2024. "Multivariate assessment of interviewer-related errors in a cross-national economic survey (Lukas Olbrich, Elisabeth Beckmann, Joseph W. Sakshaug)," Working Papers 253, Oesterreichische Nationalbank (Austrian Central Bank).
    5. Valentina Krenz & Arjen Alink & Tobias Sommer & Benno Roozendaal & Lars Schwabe, 2023. "Time-dependent memory transformation in hippocampus and neocortex is semantic in nature," Nature Communications, Nature, vol. 14(1), pages 1-17, December.
    6. Morán-Ordóñez, Alejandra & Ameztegui, Aitor & De Cáceres, Miquel & de-Miguel, Sergio & Lefèvre, François & Brotons, Lluís & Coll, Lluís, 2020. "Future trade-offs and synergies among ecosystem services in Mediterranean forests under global change scenarios," Ecosystem Services, Elsevier, vol. 45(C).
    7. Damian M. Herz & Manuel Bange & Gabriel Gonzalez-Escamilla & Miriam Auer & Keyoumars Ashkan & Petra Fischer & Huiling Tan & Rafal Bogacz & Muthuraman Muthuraman & Sergiu Groppa & Peter Brown, 2022. "Dynamic control of decision and movement speed in the human basal ganglia," Nature Communications, Nature, vol. 13(1), pages 1-15, December.
    8. Dongyan Liu & Chongran Zhou & John K. Keesing & Oscar Serrano & Axel Werner & Yin Fang & Yingjun Chen & Pere Masque & Janine Kinloch & Aleksey Sadekov & Yan Du, 2022. "Wildfires enhance phytoplankton production in tropical oceans," Nature Communications, Nature, vol. 13(1), pages 1-9, December.
    9. Stefan Pahl & Marcel P. Timmer, 2020. "Do Global Value Chains Enhance Economic Upgrading? A Long View," Journal of Development Studies, Taylor & Francis Journals, vol. 56(9), pages 1683-1705, July.
    10. Zhaogeng Yang & Yanhui Li & Peijin Hu & Jun Ma & Yi Song, 2020. "Prevalence of Anemia and its Associated Factors among Chinese 9-, 12-, and 14-Year-Old Children: Results from 2014 Chinese National Survey on Students Constitution and Health," IJERPH, MDPI, vol. 17(5), pages 1-10, February.
    11. Marco Lopez-Cruz & Fernando M. Aguate & Jacob D. Washburn & Natalia Leon & Shawn M. Kaeppler & Dayane Cristina Lima & Ruijuan Tan & Addie Thompson & Laurence Willard Bretonne & Gustavo los Campos, 2023. "Leveraging data from the Genomes-to-Fields Initiative to investigate genotype-by-environment interactions in maize in North America," Nature Communications, Nature, vol. 14(1), pages 1-14, December.
    12. Baumann, Elias & Kern, Jana & Lessmann, Stefan, 2019. "Usage Continuance in Software-as-a-Service," IRTG 1792 Discussion Papers 2019-005, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    13. repec:cup:judgdm:v:16:y:2021:i:1:p:201-237 is not listed on IDEAS
    14. C. Gabriel Hidalgo Pizango & Eurídice N. Honorio Coronado & Jhon del Águila-Pasquel & Gerardo Flores Llampazo & Johan de Jong & César J. Córdova Oroche & José M. Reyna Huaymacari & Steve J. Carver & D, 2022. "Sustainable palm fruit harvesting as a pathway to conserve Amazon peatland forests," Nature Sustainability, Nature, vol. 5(6), pages 479-487, June.
    15. Szefer Elena & Graham Jinko & Lu Donghuan & Beg Mirza Faisal & Nathoo Farouk, 2017. "Multivariate association between single-nucleotide polymorphisms in Alzgene linkage regions and structural changes in the brain: discovery, refinement and validation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 16(5-6), pages 349-365, December.
    16. Julien Collet & Samantha C Patrick & Henri Weimerskirch, 2017. "A comparative analysis of the behavioral response to fishing boats in two albatross species," Behavioral Ecology, International Society for Behavioral Ecology, vol. 28(5), pages 1337-1347.
    17. Sean Coogan & Zhixian Sui & David Raubenheimer, 2018. "Gluttony and guilt: monthly trends in internet search query data are comparable with national-level energy intake and dieting behavior," Palgrave Communications, Palgrave Macmillan, vol. 4(1), pages 1-9, December.
    18. Darcy Steeg Morris & Kimberly F. Sellers, 2022. "A Flexible Mixed Model for Clustered Count Data," Stats, MDPI, vol. 5(1), pages 1-18, January.
    19. Katrijn Delaruelle, 2023. "Migration-related inequalities in loneliness across age groups: a cross-national comparative study in Europe," European Journal of Ageing, Springer, vol. 20(1), pages 1-17, December.
    20. Christos C Ioannou & Luis Arrochela Braga Carvalho & Chessy Budleigh & Graeme D Ruxton, 2023. "Virtual prey with Lévy motion are preferentially attacked by predatory fish," Behavioral Ecology, International Society for Behavioral Ecology, vol. 34(4), pages 695-699.
    21. Vonneilich, Nico & Lüdecke, Daniel & von dem Knesebeck, Olaf, 2020. "Educational inequalities in self-rated health and social relationships – analyses based on the European Social Survey 2002-2016," Social Science & Medicine, Elsevier, vol. 267(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intell:v:101:y:2023:i:c:s0160289623000740. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/intelligence .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.