Cross-lingual unsupervised classification with multi-view transfer learning

A trained classifier system comprising:

• an encoder, comprising one or more neural network layers, that receives a text document written in a target language and converts the text document into an encoded representation;
• and
• a text classifier, comprising one or more neural network layers, that receives the encoded representation and outputs a class label for the text document;
• wherein the encoder and the text classifier of the trained classifier system were trained as part of a multi-view encoder-classifier system by performing steps comprising:
• using the encoder to generate two view representations for text documents from training documents, a first view representation comprising an encoded representation of the text document using the text document in its original language as an input into the encoder, and a second view representation comprising an encoded back-translation representation of the text document using the text document translated from its original language into another language as an input into the encoder;
• and
• using a loss to update at least the encoder and the text classifier, the loss comprising:
• a within-domain regularization related to reconstruction error of reconstructing the text document using the first view representation and a decoder;
• a cross-domain regularization related to reconstruction error of reconstructing the text document using the second view representation the decoder;
• and
• for text documents that have associated ground truth labels, a multi-view classification loss comprising a classification component related to the text classifier's predictions of the labels for those text documents and a consensus component related to reducing difference between the text classifier's predictive distribution of the first view representations and the text classifier's predictive distribution of the second view representations.

A non-transitory computer-readable medium or media comprising one or more sequences of instructions which, when executed by at least one processor, causes steps to be performed comprising:

• translating text documents in a first dataset that are written in a first language into a second language, wherein a subset of the first dataset form a labeled dataset comprising, for each text document of the subset, a label associated with text document;
• translating text documents in a second dataset that are written in the second language into the first language;
• inputting text documents, which are sampled from:
• (1) the first dataset;
• (2) the second dataset;
• (3) the first dataset translated into the second language;
• and (4) the second dataset translated into the first language into a multi-view encoder-classifier system comprising:
• an encoder that converts a text document into a latent representation;
• and
• a text classifier that receives the latent representation of a text document as an input and outputs a label for the text document;
• for text documents sampled from either the first dataset or the second dataset, determining a within-domain regularization related to reconstruction error of reconstructing the text document into its original language using the encoder and a decoder that decodes the latent representation from the encoder into a reconstruction of the text document;
• for text documents sampled from either the first dataset translated into the second language or the second dataset translated into the first language, determining a cross-domain regularization related to reconstruction error of reconstructing the text document into its original language using the translated version of the text document as the input to the encoder;
• for text documents sampled from the labeled dataset, determining a multi-view classification loss that comprises:
• (1) a classification loss component related to the text classifier's ability to correctly predict the labels for the text documents and (2) a view consensus component related to a comparison of the text classifier's label probabilities obtained using the latent representations of the text documents in the first language and the classification label probabilities obtained using the latent representations of the text document translated into the second language;
• for at least some of the text documents regardless of dataset source:
• determining an adversarial encoder loss for training the encoder to generate a latent representation for the text document that increases difficulty for a language discriminator, which receives the latent representation and outputs a prediction of whether the text document from which the latent representation was generated was in the first language or the second language, to correctly predict the language of the text document from which the latent representation was generated;
• and
• determining a language discriminator loss for training the language discriminator to correctly predict the language of the text document from which the latent representation was generated;
• using the language discriminator loss to update parameters of the language discriminator;
• and
• using the multi-view classification loss, the within-domain regularization, and the cross-domain regularization to update parameters of the encoder, the decoder, and the text classifier.