Analysis procedures for non-parametric data from performance analysis

It is vital that the reliability of a data gathering system is demonstrated clearly and in a way that is compatible with the intended analyses of the data. The data must be tested in the same way and to the same depth in which it will be processed in the subsequent analyses. In general, the work of Bland and Altman (1986) has transformed the attitude of sport scientists to testing reliability; can similar techniques be applied to the non-parametric data that most notational analysis studies generate? There are also a number of questions that inherently re-occur in these forms of data-gathering – this study aims to demonstrate practical answers to some of these questions.The most common form of data analysis in notation studies is to record frequencies of actions and their respective positions on the performance area, these are then presented as sums or totals in each respective area. What are the effects of cumulative errors nullifying each other, so that the overall totals appear less incorrect than they actually are?The application of parametric statistical techniques is often misused in notational analysis - how does this affect the confidence of the conclusions to say something about the data, with respect to more appropriate non-parametric tests? Analysing 72 research papers recently published under the banner of notational analysis, it was found that 70% did not report any reliability study and a large proportion of the remaining used questionable processes given the recent ideas in reliability testing in sports science (Atkinson, G. and Nevill, A. (1998) Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med., 26, 217–238). In some cases the reliability studies were executed on summary data, and the system was then assumed to be reliable for all of the other types of more detailed data analyses that were produced.By using practical examples from recent research and consultancy projects, undergraduate and postgraduate studies, this research investigated these issues associated with reliability studies and subsequent analyses in performance analysis, in order to give practical guidelines to enable establishing simple and reliable comparisons of non-parametric sets of data.