Depthwise separable convolutions for neural machine translation
AssigneeGoogle Inc.
InventorsAidan Nicholas Gomez, Lukasz Mieczyslaw Kaiser, Francois Chollet
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media for performing machine translation tasks. One method includes receiving an input text segment in an input language; processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers; processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output; and processing the decoder neural network output to generate a predicted output text segment in a target natural language.
Claims27
A computer implemented method for translating an input text segment in an input natural language to a corresponding output text segment in a target natural language, the method comprising:
- receiving an input text segment in the input language;
- processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers;
- and
- processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output, the generated decoder neural network output representing a predicted output text segment in the target natural language.
A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for translating an input text segment in an input natural language to a corresponding output text segment in a target natural language, the operations comprising:
- receiving an input text segment in the input language;
- processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers;
- and
- processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output, the generated decoder neural network output representing a predicted output text segment in the target natural language.
One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for translating an input text segment in an input natural language to a corresponding output text segment in a target natural language, the operations comprising:
- receiving an input text segment in the input language;
- processing the input text segment using an encoder neural network to generate an encoder neural network output, the encoder neural network comprising multiple depth wise separable convolutional neural network layers;
- and
- processing the encoder neural network output using an autoregressive decoder neural network to generate a decoder neural network output, the generated decoder neural network output representing a predicted output text segment in the target natural language.
Dependencies are inferred from the claim text.