Daphne: Data Parallelism Neural Network Simulator

In this paper we describe the guideline of Daphne, a parallel simulator for supervised recurrent neural networks trained by Backpropagation through time. The simulator has a modular structure, based on a parallel training kernel running on the CM-2 Connection Machine. The training kernel is written in CM Fortran in order to exploit some advantages of the slicewise execution model. The other modules are written in serial C code. They are used for designing and testing the network, and for interfacing with the training data. A dedicated language is available for defining the network architecture, which allows the use of linked modules.The implementation of the learning procedures is based on training example parallelism. This dimension of parallelism has been found to be effective for learning static patterns using feedforward networks. We extend training example parallelism for learning sequences with full recurrent networks. Daphne is mainly conceived for applications in the field of Automatic Speech Recognition, though it can also serve for simulating feedforward networks.