Structured text translation

A method for translating structured text using a multi-layer attention-based transformer, the method comprising:

• generating, using an encoder stored in memory, encodings from embeddings of the structured text in a first language;
• receiving, at a decoder connected to the encoder, the encodings from the encoder and embeddings from tokens in a structured translated text from a previous iteration of the multi-layer attention-based transformer, the structured translated text being in a second language different from the first language;
• and
• generating, using the decoder, a decoder output from which a plurality of tokens are determined, wherein at least one of the tokens in the plurality of tokens is included in the structured translated text.

A system for translating structured text, the system comprising:

• a multi-layer attention-based transformer comprising a plurality of layers, a layer comprising:
• an encoder configured to generate encodings from embeddings of the structured text in a first language, and
• a decoder connected to the encoder, the decoder including:
• a first multi-head attention network configured to receive tokens in a structured translated text from a previous iteration of the multi-layer attention-based transformer, the structured translated text being in a second language different from the first language,
• a second multi-head attention network configured to receive the encodings from the encoder and an output of the first multi-head attention network, and
• a feed forward network configured to receive an output of the second multi-head attention network to generate a decoder output, wherein the decoder output determines a plurality of tokens, wherein at least one of the tokens is included in the structured translated text.