Multimodal text-to-text neural machine translation using noise and domain adapters and training noise adapters while a domain adapter is frozen

A non-transitory computer-readable medium, on which is stored a computer program product comprising code instructions for executing a method for adapting a neutral machine translation (NMT) model for translating text from a noisy input in a domain comprising:

• a pretrained encoder including an attention mechanism and a feed forward layer, the encoder receiving the noisy input and generating an encoder output;
• a pretrained decoder including an attention mechanism and a feed forward layer, the decoder receiving the encoder output and generating a decoder output corresponding to translated text;
• a domain adapter injected downstream of the feed forward layer in the encoder and the decoder, the domain adapter being trained on clean in-domain data;
• and
• a plurality of individual noise adapters injected into the encoder and the decoder downstream of the feed forward layer and upstream of said domain adapter, each of the individual noise adapters being separately trained on noisy input data from a respective individual noise source for adapting to the individual noise source;
• wherein the domain adapter is trained while the pretrained encoder and the pretrained decoder are frozen;
• and
• wherein each of the individual noise adapters are trained while the pretrained encoder, the pretrained decoder, and the domain adapter are frozen.

An apparatus for adapting a pretrained neural machine translation (NMT) model for translating text from a noisy input in a domain comprising:

• a non-transitory computer-readable medium having executable instructions stored thereon for causing one or more processors and memory to:
• train a domain adapter injected into the pretrained NMT model on clean in-domain data for adapting the pretrained NMT to the domain, wherein the pretrained NMT model comprises an encoder including an encoder attention mechanism and encoder feed forward layer and a decoder including a decoder attention mechanism and decoder feed forward layer;
• wherein said training a domain adapter comprises training the pretrained NMT model with the injected domain adapter on a machine translation task while the pretrained NMT model is frozen;
• and
• separately train each of a plurality of individual noise adapters injected into the pretrained NMT model, wherein each of the individual noise adapters is respectively trained on noisy input data from an individual noise source for adapting to the individual noise source;
• wherein said separately training each of a plurality of individual noise adapters comprises, for each of the individual noise adapters, training the pretrained NMT model with the injected individual noise adapter and the trained domain adapter injected in the pretrained NMT model on a machine translation task while the pretrained NMT model and the trained domain adapter are frozen.