Systems and methods for translation evaluation

A method comprising:

• receiving training data including a source text, a context identifier for the source text, a translation text, and ground-truth translation quality information, wherein the source text comprises text displayed in a user interface of a software application and the context identifier comprises text that describes the user interface;
• generating a sequence of contextual embeddings by tokenizing the context identifier to obtain a token sequence and performing contextual embedding on the token sequence to obtain the sequence of contextual embeddings;
• generating a context vector based on the sequence of contextual embeddings, wherein the context vector represents the context identifier in a vector space;
• generating a source text representation based on the source text and the context vector using an encoder of a machine learning model, wherein the source text representation comprises a representation of the source text and a visual context of the source text based on the context identifier;
• generating a translation text representation based on the translation text using the encoder;
• generating a translation quality prediction score for the translation text based on the source text representation and the translation text representation using a decoder of the machine learning model, wherein the translation quality prediction score accounts for the visual context of the source text within the user interface;
• computing a loss function based on the predicted translation quality information and the ground-truth translation quality information;
• and
• updating parameters of the machine learning model based on the loss function.

A system comprising:

• a memory component;
• and
• a processing device coupled to the memory component, the processing device configured to perform operations comprising:
• receiving a source text, a context identifier for the source text, and a translation text, wherein the source text comprises text displayed in a user interface of a software application and the context identifier comprises text that describes the user interface;
• generating a sequence of contextual embeddings by tokenizing the context identifier to obtain a token sequence and performing contextual embedding on the token sequence to obtain the sequence of contextual embeddings;
• generating a context vector based on the sequence of contextual embeddings, wherein the context vector represents the context identifier in a vector space;
• generating a source text representation based on the source text and the context vector using an encoder of a machine learning model, wherein the source text representation comprises a representation of the source text and a visual context of the source text based on the context identifier;
• generating a translation text representation based on the translation text using the encoder;
• and
• generating a translation quality prediction score for the translation text based on the source text representation and the translation text representation using a decoder of the machine learning model, wherein the translation quality prediction score accounts for the visual context of the source text within the user interface.