US 2025/0094704 A1 · LLM FINE-TUNING FOR TEXT SUMMARIZATION

Abstract

Systems, methods, and other embodiments associated with automated fine-tuning of text summarization for large language models are described herein. In one embodiment, a method accesses a collection of text samples. The text samples include a body of text and an example summary. The method fine-tunes a large language model (LLM) based on a loss function that compares the example summary and a generated summary generated by the LLM. The example and generated summaries are compared at sentence, paragraph, and/or article levels. The method generates an evaluation score for performance of the tuned LLM as a text summarizer based on a further comparison of a reference summary and a summary generated by the tuned LLM. The method then automatically determines to deploy the tuned LLM to a text summarization task in response to the evaluation score satisfying a threshold.

Claims20

Dependent claims

One or more non-transitory computer-readable media that include stored thereon computer-executable instructions that when executed by at least a processor of a computer system cause the computer system to:

access a collection of text samples, wherein the text samples include a body of text in natural language and an example summary in natural language of the body of text;
fine-tune a large language model based on a loss function that compares the example summary and a generated summary generated by the large language model, wherein the example summary and the generated summary are compared at more than one of a sentence level, a paragraph level, and an article level;
generate an evaluation score for performance of the tuned large language model as a text summarizer based on a second comparison of a reference summary and a second generated summary generated by the tuned large language model;
and
automatically signal that the fine tuning of the tuned large language model is complete in response to the evaluation score satisfying a threshold.

2 more elements

A computer-implemented method, comprising:

accessing a collection of text samples, wherein the text samples include a body of text in natural language and an example summary in natural language of the body of text;
fine-tuning a large language model based on a loss function that compares words shared between the example summary and a generated summary generated by the large language model;
generating an evaluation score for performance of the tuned large language model as a text summarizer based on a second comparison of words shared between a reference summary and a second generated summary generated by the tuned large language model;
and
automatically determining to deploy the tuned large language model to a text summarization task in response to the evaluation score satisfying a threshold.

2 more elements

A computing system, comprising:

at least one processor connected to at least one memory;
one or more non-transitory computer-readable media that include stored thereon computer-executable instructions that when executed by at least the processor accessing the memory cause the computing system to:
access a collection of text samples, wherein the text samples include a body of text in natural language and an example summary in natural language of the body of text;
retrain a large language model based on a loss function that compares words shared between the example summary and a generated summary generated by the large language model;
generate an evaluation score for performance of the tuned large language model as a text summarizer based on a second comparison of words shared between a reference summary and a second generated summary generated by the tuned large language model;
and
automatically determine to deploy the tuned large language model to a text summarization task in response to the evaluation score satisfying a threshold.

4 more elements

Dependencies are inferred from the claim text.

Llm fine-tuning for text summarization

Life of this patent

Llm fine-tuning for text summarization

Life of this patent

Abstract

Claims20