US 12,755,098 B2Granted

Task attention mechanism for context window augmentation

AssigneeInternational Business Machines Corporation

InventorsZhong Fang Yuan, Chi Nan, Tong Liu, Li Juan Gao

Life of this patent

Estimating the term…

Loading patent…

Abstract

One or more computer-implemented methods, computer systems and/or computer program products of use provided herein relate to employing a task attention mechanism for context window augmentation of a large language model (LLM). In various embodiments, a computer-implemented method comprises receiving, at a machine learning model, first text data. The computer-implemented method further comprises determining, via the machine learning model, a token length of the first text data. The computer-implemented method further comprises in response to determining that the token length is greater than a threshold token length of a first language machine learning model, generating via the machine learning model a first logical graph from the first text data, wherein the first logical graph incorporates tokens exceeding the threshold token length of the first language machine learning model without a portion of the first text data becoming eliminated, and the first logical graph is incorporated within the machine learning model.

Claims15

Dependent claims

A computer-implemented method comprising:

receiving, by a processor executing a first language machine learning model, first text data;
determining, by the processor, that a token length of the first text data exceeds a predefined maximum token length supported by the first language machine learning model;
in response to determining that the token length exceeds the predefined maximum token length, automatically transforming, by the processor, the first text data into a first logical graph representation that:
(i) represents all tokens of the first text data as nodes and logical relationships between the tokens as edges,
(ii) preserves semantic relationships of the first text data without truncating or eliminating any portion of the first text data, and
(iii) converts textual sequence distances into logical graph distances independent of token length constraints;
storing the first logical graph representation in computer memory accessible to the first language machine learning model as a structured internal knowledge representation;
and
in response to receiving a query, generating a second logical graph representation from the query and performing graph structure matching between the second logical graph representation and the first logical graph representation to identify a matching subgraph used to generate a response via the first language machine learning model.

6 more elements

A computer system, comprising:

a processor set;
a set of one or more computer-readable storage media storing program instructions, a first language machine learning model executable by the processor set, to cause the processor set to perform computer operations comprising:
receiving first text data;
determining that a token length of the first text data exceeds a maximum token length supported by the first language machine learning model;
in response to the token length exceeding the maximum token length, invoking a graph construction component to generate a first logical graph representation of the first text data, the first logical graph representation:
(i) comprising nodes corresponding to tokens of the first text data and edges corresponding to logical relationships between the tokens,
(ii) preserving semantic content of the first text data without truncating or eliminating any portion of the first text data, and
(iii) representing relational proximity between portions of the first text data independently of token position limits;
storing the first logical graph representation in a graph memory region distinct from a token context memory of the first language machine learning model;
and
responsive to receipt of a query, accessing the graph memory region to retrieve one or more logically related portions of the first logical graph representation for use by the first language machine learning model in generating a response.

8 more elements

Dependencies are inferred from the claim text.