Compressing Information Provided to a Machine-Trained Generative Model

A computing system for performing abstractive summarization, comprising:

• a data store for storing computer-readable instructions;
• a processing system for executing the computer-readable instructions in the data store based, to perform operations including:
• receiving original input information, the original input information including a first plurality of sentences;
• performing word-level encoding of the original input information using a first part of a machine-trained transformer model, to provide word-level encoded information;
• performing sentence-level encoding of the word-level encoded information using a second part of the machine-trained transformer model, to provide scores associated with the first plurality of sentences, each score identifying a relevance of a corresponding sentence to an interpretation of the original input information, wherein model weights used in the first part and the second part of the machine-trained transformer model are trained together;
• selecting a subset of the first plurality of sentences in the original input information based on the scores, to provide modified input information having a second plurality of sentences, the second plurality of sentences having fewer sentences than the first plurality of sentences;
• and
• performing abstractive summarization of the modified input information using a machine-trained generative model to reduce a size of the modified input information, the abstractive summarization involving changing a wording of at least one sentence in the modified input information.

A computer-readable storage medium for storing computer-readable instructions, a processing system executing the computer-readable instructions to perform operations, the operations comprising:

• receiving original input information, the original input information including a first plurality of sentences;
• partitioning the original input information into a plurality of windows, each window having a given length that encompasses a part of the original input information, and each window that has a preceding window overlapping the preceding window by a prescribed amount of information;
• performing word-level encoding by converting the plurality of windows to a plurality of instances of word-level encoded information in parallel;
• performing sentence-level encoding by converting the plurality of instances of word-level encoded information to a plurality of scores, each score identifying a relevance of a corresponding sentence to an interpretation of the original input information, wherein model weights used in the performing word-level encoding and performing sentence-level encoding are trained together;
• selecting a subset of the first plurality of sentences in the original input information based on the scores, to provide modified input information having a second plurality of sentences, the second plurality of sentences having fewer sentences than the first plurality of sentences;
• and
• providing the modified input information to a machine-trained generative model, the machine-trained generative model being trained to convert the modified input information into output information that is consumable by an output device.