Longitudinal Evaluation of Food Safety Discussion-Based Exercises: Tool Development and Initial Validation

This study was undertaken for three major purposes: first, to provide initial validation data on an assessment tool designed to provide evidence supporting the following two purposes; second, to analyze the lessons learned process from exercise conduct through subsequent organizational action; and third, to investigate the long-term impact of discussion-based exercises associated with improving food safety preparedness. The major hypothesis forming the basis of the investigation was a logic model containing five sets of variables selected from relevant literature on emergency/disaster preparedness, training evaluation, and instructional design. One variable set, Organizational Actions, served as the criterion of interest. The other four variables served as predictor variables; these were: a) Exercises Outcomes; b) Exercise Design Features; c) Organizational and Personal Characteristics, and d) Demographic Characteristics. The target exercises selected for investigation and evaluation were a class of discussion-based activities conducted for various audiences of food safety personnel from government, industry, public health, and academia. The four exercise sessions from which participants were selected had occurred between two years and six months previously. There were two stages of data collection-personal interviews to assure data relevance and an online survey. Fourteen participants were interviewed and 47 complete responses were obtained from the survey. Data analysis provided considerable support for assessment tool validity. First, it would appear that discussion-based exercises can have some positive long-term impact on organizational change, a finding that heretofore, has not been demonstrated in the relevant research literature. Second, empirical evidence was found for the logic model, with almost 60% of the variance in the criterion variable accounted for by the other measures. Third, given that the exercises were designed to promote coordination and collaboration across agencies and industry sectors, it was found that by tailoring the exercise to participant backgrounds and designing activities to promote long-term learning, the potential for cross-organizational action can be increased. Other conclusions highlighted specific recommendations that could improve exercise design.