Language model summarization using semantical clustering

A method comprising:

• identifying a plurality of concepts reflected in text data;
• generating a plurality of concept clusters based on similarity among the plurality of concepts;
• for each concept cluster of the plurality of concept clusters:
• based on a size of text, in the text data, that corresponds to the concepts in said each concept cluster, determining a particular size;
• generating a prompt that indicates the particular size;
• sending the prompt to a language model, causing the language model to generate a summary of the text corresponding to the concepts in said each concept cluster;
• generating a summary response of the text data by aggregating the summary of each concept cluster of the plurality of concept clusters;
• wherein the method is performed by one or more computing devices.

One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause:

• identifying a first plurality of concepts reflected in text data;
• generating a first plurality of concept clusters based on similarity among the first plurality of concepts;
• causing a first language model to generate a summary based on the text data;
• identifying a second plurality of concepts reflected in the summary;
• generating a second plurality of concept clusters based on similarity among the second plurality of concepts;
• determining whether the second plurality of concept clusters is missing one or more concept clusters from the first plurality of concept clusters.

One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause:

• identifying a plurality of concepts reflected in text data;
• generating a plurality of concept clusters based on similarity among the plurality of concepts;
• for each concept cluster of the plurality of concept clusters:
• based on a size of text, in the text data, that corresponds to the concepts in said each concept cluster, determining a particular size;
• generating a prompt that indicates the particular size;
• sending the prompt to a language model, causing the language model to generate a summary of the text corresponding to the concepts in said each concept cluster;
• generating a summary response of the text data by aggregating the summary of each concept cluster of the plurality of concept clusters.