# US 2022/0277135 A1 · Systems and methods for query-focused summarization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0277135 A1
- Document ID: US20220277135A1
- Status: Published application
- Kind: A1
- Application number: 17/749,837
- Filed: 2022-05-20
- Published: 2022-09-01
- Earliest claimed: 2016-02-01
- Assignees: Salesforce, Inc.
- Applicants: Salesforce, Inc.
- Inventors: Wojciech Kryscinski; Alexander R. Fabbri; Jesse Vig
- Web page: https://patentgrep.com/patent/US20220277135A1

## CPC

- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 16/3329: Natural language query formulation
- G06F 16/345: {Summarisation for human users}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Embodiments described herein provide a query-focused summarization model that employs a single or dual encoder model. A two-step approach may be adopted that first extracts parts of the source document and then synthesizes the extracted segments into a final summary. In another embodiment, an end-to-end approach may be adopted that splits the source document into overlapping segments, and then concatenates encodings into a single embedding sequence for the decoder to output a summary.

## Claims

### Claim 1

A method for query-focused summarization, the method comprising:
receiving, via a communication interface, a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
dividing the source document into a plurality of source passages;
encoding, via an encoder model, a source passage from the plurality of source passages and the input query separated by a delimiter;
generating, by a final layer of the encoder model, a predicted relevance between the source passage and the reference summary based on the encoding;
computing a loss function based on the predicted relevance; and
updating the encoder model based on the computed loss function.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
receiving a testing query and a testing source document;
generating, via the encoder model, predicted relevance scores corresponding to parts of the testing source document;
extracting at least one part of the testing source document relevant to the testing query based on the predicted relevance scores; and
synthesizes, via an abstractor model, the extracted at least one part into a final summary of the testing source document.

### Claim 3 (depends on 1)

The method of claim 1, wherein the loss function is a mean square value of a number of predicted relevances.

### Claim 4 (depends on 1)

The method of claim 1, wherein the encoder model is a single-encoder model that jointly encodes the source passage appended with the input query.

### Claim 5 (depends on 1)

The method of claim 1, wherein the encoder model is a double-encoder model that separately encodes the input query and the source passage into a query embedding and a passage embedding, respectively.

### Claim 6 (depends on 5)

The method of claim 5, wherein the generating the predicted relevance further comprises calculating a cosine similarity between the query embedding and the passage embedding.

### Claim 7 (depends on 5)

The method of claim 5, wherein the passage embedding is pre-computed and stored for a given testing query.

### Claim 8

A system for query-focused summarization, the system comprising:
a communication interface that receives a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
a memory storing a plurality of processor-executable instructions; and
a processor executing the instructions to perform operations comprising:
dividing the source document into a plurality of source passages;
encoding, via an encoder model, a source passage from the plurality of source passages and the input query separated by a delimiter;
generating, by a final layer of the encoder model, a predicted relevance between the source passage and the reference summary based on the encoding;
computing a loss function based on the predicted relevance; and
updating the encoder model based on the computed loss function.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise:
receiving a testing query and a testing source document;
generating, via the encoder model, predicted relevance scores corresponding to parts of the testing source document;
extracting at least one part of the testing source document relevant to the testing query based on the predicted relevance scores; and
synthesizes, via an abstractor model, the extracted at least one part into a final summary of the testing source document.

### Claim 10 (depends on 8)

The system of claim 8, wherein the loss function is a mean square value of a number of predicted relevances.

### Claim 11 (depends on 8)

The system of claim 8, wherein the encoder model is a single-encoder model that jointly encodes the source passage appended with the input query.

### Claim 12 (depends on 8)

The system of claim 8, wherein the encoder model is a double-encoder model that separately encodes the input query and the source passage into a query embedding and a passage embedding, respectively.

### Claim 13 (depends on 12)

The system of claim 12, wherein an operation of generating the predicted relevance further comprises calculating a cosine similarity between the query embedding and the passage embedding.

### Claim 14 (depends on 13)

The system of claim 13, wherein the passage embedding is pre-computed and stored for a given testing query.

### Claim 15

A system for query-focused summarization, the system comprising:
a communication interface that receives a source document, an input query posing a question on a content of the source document and a reference summary of the source document;
a memory storing a plurality of processor-executable instructions; and
a processor executing the instructions to perform operations comprising:
splitting the source document into a plurality of overlapping segments;
encoding, via an encoder, each overlapping segment separately appended with the input query into a respective encoding;
concatenating encoded segments from the plurality of overlapping segments into an embedding sequence; and
generating, via a decoder, an output summary from the embedding sequence.

### Claim 16 (depends on 15)

The system of claim 15, wherein an operation of encoding comprises:
applying sparse attention within each overlapping segment without cross-attention between encoded segments.

### Claim 17 (depends on 15)

The system of claim 15, wherein an operation of generating the output summary comprises:
attending, by the decoder, to all encoded segments jointly.

### Claim 18 (depends on 15)

The system of claim 15, wherein the encoder and the decoder are built on a Transformer model.

### Claim 19 (depends on 15)

The system of claim 15, wherein the source document contains query-relevant sections that are related to the input query, and
wherein the query-relevant sections have a size within a processing capacity of a Transformer model.
