System and method for combining language models with natural language processing for summarization

A system comprising:

• a memory having computer-readable instructions stored thereon;
• and
• a processor that executes the computer-readable instructions to:
• receive a set of documents for a prompt being input to a language model having a context window having a token limit, wherein a token count of the set of documents requires a first amount of computer memory;
• reduce the token count of the set of documents to fit within the context window of the language model by:
• inputting the set of documents into at least one information extraction model;
• executing each of the at least one information extraction model to extract a first plurality of text segments from the set of documents, wherein the first plurality of text segments reduces a number of tokens from a total number of tokens in each document of the set of documents to a first token count, and wherein the first plurality of text segments are usable as inputs to determine a trend across the set of documents;
• determining a second plurality of text segments based on the first plurality of text segments, wherein the second plurality of text segments correspond to documents selected from the set of documents based on one or more of at least one keyword, metadata, or a topic of interest, and wherein a number of the second plurality of text segments is less than or equal to a number of the first plurality of text segments;
• determining a third plurality of text segments from the second plurality of text segments based on at least one of a frequency of occurrence, a priority score, or a location of a text segment in a document of the set of documents, wherein a number of the third plurality of text segments is less than or equal to the number of the second plurality of text segments, and wherein the third plurality of text segments reduces the number of tokens to a second token count that is less than the first token count;
• in response to reducing the token count, generate a compressed representation of the set of documents from the third plurality of text segments to include in a compressed prompt for the language model, wherein
• the compressed prompt comprises the second token count stored in a second amount of computer memory that is less than the first amount of computer memory associated with the token count of the set of documents, and wherein the compressed representation being generated by concatenating the third plurality of text segments from the second plurality of text second and excluding unselected text segments from the set of documents that are associated with the trend;
• input the compressed prompt into the language model;
• and
• execute the language model to generate the summary based on the compressed prompt for the set of documents.

A method comprising:

• receiving, by a processor executing computer-readable instructions stored on a memory, a set of documents for a prompt being input to a language model having a context window having a token limit, wherein a token count of the set of documents requires a first amount of computer memory;
• reducing the token count of the set of documents to fit within the context window of the language model by:
• inputting, by the processor, the set of documents into at least one information extraction model;
• executing, by the processor, each of the at least one information extraction model to extract a first plurality of text segments from the set of documents, wherein the first plurality of text segments reduces a number of tokens from a total number of tokens in each document of the set of documents to a first token count, and wherein the first plurality of text segments are usable as inputs to determine a trend across the set of documents;
• determining, by the processor, a second plurality of text segments based on the first plurality of text segments, wherein the second plurality of text segments correspond to documents selected from the set of documents based on one or more of at least one keyword, metadata, or a topic of interest, and wherein a number of the second plurality of text segments is less than or equal to a number of the first plurality of text segments;
• determining, by the processor, a third plurality of text segments from the second plurality of text segments based on at least one of a frequency of occurrence, a priority score, or a location of a text segment in a document of the set of documents, wherein a number of the third plurality of text segments is less than or equal to the number of the second plurality of text segments, and wherein the third plurality of text segments reduces the number of tokens to a second token count that is less than the first token count;
• in response to reducing the token count, generating, by the processor, a compressed representation of the set of documents from the third plurality of text segments to include in a compressed prompt for the language model, wherein the compressed prompt comprises the second token count stored in a second amount of computer memory that is less than the first amount of computer memory associated with the token count of the set of documents, and wherein the compressed representation being generated by concatenating the third plurality of text segments from the second plurality of text second and excluding unselected text segments from the set of documents that are associated with the trend;
• inputting, by the processor, the compressed prompt into the language model;
• and
• executing, by the processor, the language model to generate the summary based on the compressed prompt for the set of documents.

A non-transitory computer-readable medium comprising computer-readable instructions stored thereon that when executed by a processor cause the processor to:

• receive a set of documents to generate a summary summarizing a trend across the set of documents;
• input the set of documents into at least one information extraction model;
• execute each of the at least one information extraction model to extract a first plurality of text segments from the set of documents, wherein the first plurality of text segments reduces a number of tokens from a total number of tokens in each document of the set of documents to a first token count, and wherein the first plurality of text segments are usable as inputs to determine a trend across the set of documents;
• determine a second plurality of text segments based on the first plurality of text segments, wherein the second plurality of text segments correspond to documents selected from the set of documents based on one or more of at least one keyword, metadata, or a topic of interest, and wherein a number of the second plurality of text segments is less than or equal to a number of the first plurality of text segments;
• determine a third plurality of text segments from the second plurality of text segments based on at least one of a frequency of occurrence, a priority score, or a location of a text segment in a document of the set of documents, wherein a number of the third plurality of text segments is less than or equal to the number of the second plurality of text segments, and wherein the third plurality of text segments reduces the number of tokens to a second token count that is less than the first token count;
• compute the priority score for each of the second plurality of text segments, wherein computing the priority score includes using Priority Score=Frequency X Term Frequency-Inverse Document Frequency Score, where Priority Score is the computed priority score for the text segment, Frequency is a number of times the text segment occurs across the second plurality of text segments, and Term Frequency-Inverse Document Frequency Score is a score indicating a relevance of the text segment in the set of documents;
• select at least a subset of the text segments from the second plurality of text segments whose priority score is greater than a predetermined threshold;
• generate a compressed representation of the set of documents from the third plurality of text segments to include in a prompt for a language model, wherein the prompt comprises the second token count;
• input the prompt into the language model;
• and
• execute the language model to generate the summary based on the compressed prompt for the set of documents.

A non-transitory computer-readable medium comprising computer-readable instructions stored thereon that when executed by a processor cause the processor to:

• receive a set of documents to generate a summary summarizing a trend across the set of documents;
• input the set of documents into at least one information extraction model;
• execute each of the at least one information extraction model to extract a first plurality of text segments from the set of documents, wherein the first plurality of text segments reduces a number of tokens from a total number of tokens in each document of the set of documents to a first token count, and wherein the first plurality of text segments are usable as inputs to determine a trend across the set of documents;
• determine a second plurality of text segments based on the first plurality of text segments, wherein the second plurality of text segments correspond to documents selected from the set of documents based on one or more of at least one keyword, metadata, or a topic of interest, and wherein a number of the second plurality of text segments is less than or equal to a number of the first plurality of text segments;
• determine a third plurality of text segments from the second plurality of text segments based on at least one of a frequency of occurrence, a priority score, or a location of a text segment in a document of the set of documents, wherein a number of the third plurality of text segments is less than or equal to the number of the second plurality of text segments, and wherein the third plurality of text segments reduces the number of tokens to a second token count that is less than the first token count;
• compute the priority score for each of the second plurality of text segments, wherein computing the priority score includes using Priority Score=Frequency X Domain-Specific Term Count, where Priority Score is the computed priority score for the text segment, Frequency is a number of times the text segment occurs across the second plurality of text segments, and Domain-Specific term Count is a number of domain-specific terms in the text segment;
• select at least a subset of the text segments from the second plurality of text segments whose priority score is greater than a predetermined threshold;
• generate a compressed representation of the set of documents from the third plurality of text segments to include in a prompt for a language model, wherein the prompt comprises the second token count;
• input the prompt into the language model;
• and
• execute the language model to generate the summary based on the compressed prompt for the set of documents.