Collaborative Data Management for Longitudinal Studies

Efficient data cleaning and management is critical to the success of any large research project. This is particularly true in the case of longitudinal studies and/or those in which the data management tasks are shared among many individuals. Faced with several such projects, we developed a flexible, easy-to-use system for cleaning and managing research datasets. The system is modular, making it easy for different individuals to work on different parts of the process. This modularity also permits substantial code reuse over multiple waves of a longitudinal study. A central focus of the system is the idea of data testing; users write tests for specific variables that may then be rerun when a new wave of data becomes available or when changes to the data have been made. Although the basic ideas could be implemented in any statistical package or programming language, Stata is particularly well-suited to the task. In addition, we have written an ado-file to automate the process of building a data set and another to generate basic tests automatically from an existing dataset. Although the system was designed for use by large, collaborative projects, individuals can also benefit from using it for personal research projects.