Rectifying Ill-Formed Interlingual Space: A Framework for Zero-Shot Translation on Modularized Multilingual NMT

The multilingual neural machine translation (NMT) model can handle translation between more than one language pair. From the perspective of industrial applications, the modularized multilingual NMT model (M2 model) that only shares modules between the same languages is a practical alternative to the model that shares one encoder and one decoder (1-1 model). Previous works have proven that the M2 model can benefit from multiway training without suffering from capacity bottlenecks and exhibits better performance than the 1-1 model. However, the M2 model trained on English-centric data is incapable of zero-shot translation due to the ill-formed interlingual space. In this study, we propose a framework to help the M2 model form an interlingual space for zero-shot translation. Using this framework, we devise an approach that combines multiway training with a denoising autoencoder task and incorporates a Transformer attention bridge module based on the attention mechanism. We experimentally show that the proposed method can form an improved interlingual space in two zero-shot experiments. Our findings further extend the use of the M2 model for multilingual translation in industrial applications.