Contextual text generation for question answering and text summarization with supervised representation disentanglement and mutual information minimization

A system for disentangled data generation, comprising:

• a hardware processor;
• and
• a memory that stores:
• a dataset including a plurality of pairs, each formed from a given one of a plurality of input text structures and a given one of a plurality of style labels for the plurality of input text structures;
• and
• computer program code which, when executed by the hardware processor, implements:
• training code that trains an encoder neural network to disentangle a sequential text input into disentangled representations, including a content embedding and a style embedding, based on a subset of the dataset, using an objective function that includes a regularization term that minimizes mutual information between the content embedding and the style embedding, and that trains a generator neural network to generate a text output that includes content from the style embedding, expressed in a style other than that represented by the style embedding of the text input, wherein the objective function is:
• VAE+λreg
• wherein λ is a hyperparameter reweighting a regularization reg and a variational autoencoder objective VAE, where reg is expressed as Dis+MI(s;
• c), including a disentanglement loss Dis and a mutual information term MI(s;
• c) base on a style embedding s and a content embedding c.