Depthwise separable convolutions for neural machine translation

A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for translating an input text segment in an input natural language to a corresponding output text segment in a target natural language, the operations comprising:

• receiving an input text segment in the input language;
• processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers;
• and
• processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output, the generated decoder neural network output representing a predicted output text segment in the target natural language.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for translating an input text segment in an input natural language to a corresponding output text segment in a target natural language, the operations comprising:

• receiving an input text segment in the input language;
• processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers;
• and
• processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output, the generated decoder neural network output representing a predicted output text segment in the target natural language.