IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v44y2019i6p706-732.html
   My bibliography  Save this article

Process Data in NAEP: Past, Present, and Future

Author

Listed:
  • Yoav Bergner

    (New York University)

  • Alina A. von Davier

    (ACTNext by ACT)

Abstract

This article reviews how National Assessment of Educational Progress (NAEP) has come to collect and analyze data about cognitive and behavioral processes (process data) in the transition to digital assessment technologies over the past two decades. An ordered five-level structure is proposed for describing the uses of process data. The levels in this hierarchy range from ignoring the processes (i.e., counting only the outcomes), to incorporating process data as auxiliary or essential in addition to the outcome, to modeling the process as the outcome itself, either holistically in a rubric score or in a measurement model that accounts for sequential dependencies. Historical examples of these different uses are described as well as recent results using nontraditional analytical approaches. In the final section, speculative future directions incorporating state-of-the-art technologies and analysis methods are described with an eye toward hard-to-measure constructs such as higher order problem-solving and collaboration.

Suggested Citation

  • Yoav Bergner & Alina A. von Davier, 2019. "Process Data in NAEP: Past, Present, and Future," Journal of Educational and Behavioral Statistics, , vol. 44(6), pages 706-732, December.
    • Handle: RePEc:sae:jedbes:v:44:y:2019:i:6:p:706-732
    DOI: 10.3102/1076998618784700
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998618784700
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998618784700?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Jeffrey Mo, 2017. "Collaborative problem solving," PISA in Focus 78, OECD Publishing.
    2. Gabadinho, Alexis & Ritschard, Gilbert & Müller, Nicolas S & Studer, Matthias, 2011. "Analyzing and Visualizing State Sequences in R with TraMineR," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 40(i04).
    3. Duncan McVicar & Michael Anyadike‐Danes, 2002. "Predicting successful and unsuccessful transitions from school to work by using sequence methods," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 165(2), pages 317-334, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pujue Wang & Hongyun Liu, 2024. "Polytomous Effectiveness Indicators in Complex Problem-Solving Tasks and Their Applications in Developing Measurement Model," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 877-902, September.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. repec:jss:jstsof:40:i04 is not listed on IDEAS
    2. Erofili Grapsa & Dorrit Posel, 2016. "Sequencing the real time of the elderly: Evidence from South Africa," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 35(25), pages 711-744.
    3. Piccarreta, Raffaella & Bonetti, Marco, 2019. "Assessing and comparing models for sequence data by microsimulation (with Supplementary Material)," SocArXiv 3mcfp, Center for Open Science.
    4. Keefe Murphy & T. Brendan Murphy & Raffaella Piccarreta & I. Claire Gormley, 2021. "Clustering longitudinal life‐course sequences using mixtures of exponential‐distance models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(4), pages 1414-1451, October.
    5. Kelly Huegaerts & Martin Wagener & Christophe Vanroelen, 2020. "Is Mental Health a Predictor for a Smooth School-to-Work-Transition? A 20-Month Follow-Up Study of Brussels Youth," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 15(5), pages 1549-1567, November.
    6. Francisco Rowe & Jonathan Corcoran & Martin Bell, 2017. "The returns to migration and human capital accumulation pathways: non-metropolitan youth in the school-to-work transition," The Annals of Regional Science, Springer;Western Regional Science Association, vol. 59(3), pages 819-845, November.
    7. Mike Smet & Barbara Janssens, 2015. "Educational Pathways of students who enrolled in a subject-specific teacher training in Flanders: An Optimal Matching Approach," EcoMod2015 8577, EcoMod.
    8. Marco Raffaella Piccarreta & Marco Bonetti & Stefano Lombardi, 2018. "Comparing models for sequence data: prediction and dissimilarities," Working Papers 113, "Carlo F. Dondena" Centre for Research on Social Dynamics (DONDENA), Università Commerciale Luigi Bocconi.
    9. Tomas Berglund & Roy A Nielsen & Olof Reichenberg & Jørgen Svalund, 2023. "Temporary Contracts, Employment Trajectories and Dualisation: A Comparison of Norway and Sweden," Work, Employment & Society, British Sociological Association, vol. 37(2), pages 505-524, April.
    10. Struffolino, Emanuela, 2019. "Navigating the early career: The social stratification of young workers’ employment trajectories in Italy," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 63, pages 1-17.
    11. Nicholas Longford & Ioana C. Salagean, 2013. "A study of the labour market trajectories in the Grand Duchy of Luxembourg," Economics Working Papers 1396, Department of Economics and Business, Universitat Pompeu Fabra.
    12. Marcel Raab & Emanuela Struffolino, 2020. "The Heterogeneity of Partnership Trajectories to Childlessness in Germany," European Journal of Population, Springer;European Association for Population Studies, vol. 36(1), pages 53-70, March.
    13. Ekaterina Mitrofanova & Alyona Artamonova, 2016. "The perspectives of family policy in Russia amid increasing cohabitation," European Journal of Government and Economics, Europa Grande, vol. 5(1), pages 47-63, June.
    14. Baccar, Mariem & Raynal, Hélène & Sekhar, Muddu & Bergez, Jacques-Eric & Willaume, Magali & Casel, Pierre & Giriraj, P. & Murthy, Sanjeeva & Ruiz, Laurent, 2023. "Dynamics of crop category choices reveal strategies and tactics used by smallholder farmers in India to cope with unreliable water availability," Agricultural Systems, Elsevier, vol. 211(C).
    15. Sebastian Brumann & Ulrike Ohl & Johannes Schulz, 2022. "Inquiry-Based Learning on Climate Change in Upper Secondary Education: A Design-Based Approach," Sustainability, MDPI, vol. 14(6), pages 1-29, March.
    16. Cees H. Elzinga, 2010. "Complexity of Categorical Time Series," Sociological Methods & Research, , vol. 38(3), pages 463-481, February.
    17. Júlia Mikolai & Hill Kulu, 2019. "Union dissolution and housing trajectories in Britain," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 41(7), pages 161-196.
    18. Joanne S. Muller & Nicole Hiekel & Aart C. Liefbroer, 2020. "The Long-Term Costs of Family Trajectories: Women’s Later-Life Employment and Earnings Across Europe," Demography, Springer;Population Association of America (PAA), vol. 57(3), pages 1007-1034, June.
    19. Zachary Van Winkle & Anette Fasang, 2021. "The complexity of employment and family life courses across 20th century Europe: More evidence for larger cross-national differences but little change across 1916‒1966 birth cohorts," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 44(32), pages 775-810.
    20. Moehring, Katja & Weiland, Andreas & Reifenscheid, Maximiliane & Naumann, Elias & Wenz, Alexander & Rettig, Tobias & Krieger, Ulrich & Fikel, Marina & Cornesse, Carina & Blom, Annelies G., 2021. "Inequality in employment trajectories and their socio-economic consequences during the early phase of the COVID-19 pandemic in Germany," SocArXiv m95df, Center for Open Science.
    21. Babette Bühler & Katja Möhring & Andreas P. Weiland, 2022. "Assessing dissimilarity of employment history information from survey and administrative data using sequence analysis techniques," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(6), pages 4747-4774, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:44:y:2019:i:6:p:706-732. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.