On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Neural machine translation is a relatively new approach to statisticalmachine translation based purely on neural networks. The neural machinetranslation models often consist of an encoder and a decoder. The encoderextracts a fixed-length representation from a variable-length input sentence,and the decoder generates a correct translation from this representation. Inthis paper, we focus on analyzing the properties of the neural machinetranslation using two models; RNN Encoder–Decoder and a newly proposed gatedrecursive convolutional neural network. We show that the neural machinetranslation performs relatively well on short sentences without unknown words,but its performance degrades rapidly as the length of the sentence and thenumber of unknown words increase. Furthermore, we find that the proposed gatedrecursive convolutional network learns a grammatical structure of a sentenceautomatically.

Further reading#

• Access Paper in arXiv.org