# US 2026/0064972 A1 · Language-Based Attention Mechanisms for Machine-Learned Models

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0064972 A1
- Document ID: US20260064972A1
- Status: Published application
- Kind: A1
- Application number: 18/816,835
- Filed: 2024-08-27
- Published: 2026-03-05
- Earliest claimed: 2024-08-27
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Ramin Mehran; Nilpa Jha
- Web page: https://patentgrep.com/patent/US20260064972A1

## CPC

- G06F 40/30: Semantic analysis
- G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking
- G06N 3/045: Combinations of networks
- G06N 20/00: Machine learning
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

For each portion of a content item, the portion of the content item can be processed with a machine-learned Large Foundational Model (LFM) to obtain an attentional value output comprising a summarization of the portion. The attentional value output can be processed with the machine-learned LFM to obtain an attentional query output descriptive of thematic elements associated with the portion, and an attentional key output comprising key words and/or phrases from the portion. An attentional weight can be determined for each portion based on a semantic similarity between the attentional query output and the attentional key output for each portion. A subset of portions of the content item can be selected based on the attentional weight determined for the subset. A task output can be generated based on the attentional value output obtained for each of the subset of portions.

## Claims

### Claim 1

A computer-implemented method, comprising:
for each portion of a plurality of portions of a content item:
processing, by a computing system comprising one or more processing devices, the portion of the content item with a machine-learned Large Foundational Model (LFM) to obtain an attentional value output comprising a summarization of the portion of the content item;
processing, by the computing system, the attentional value output with the machine-learned LFM to obtain an attentional query output descriptive of thematic elements associated with the portion of the content item; and
processing, by the computing system, the attentional value output with the machine-learned LFM to obtain an attentional key output comprising key words and/or phrases from the portion of the content item;
determining, by the computing system, an attentional weight for each portion of the plurality of portions of the content item based on a semantic similarity between the attentional query output and the attentional key output for each of the plurality of portions;
selecting, by the computing system, a subset of portions of the plurality of portions of the content item based on the attentional weight determined for each of the subset of portions; and
generating, by the computing system, a task output based on the attentional value output obtained for each of the subset of portions.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein processing the attentional value output with the machine-learned LFM to obtain the attentional query output comprises:
processing, by the computing system, the attentional value output with a first instance of the machine-learned LFM to obtain the attentional query output;
wherein processing the attentional value output with the machine-learned LFM comprises:
processing, by the computing system, the attentional value output with a second instance of the machine-learned LFM to obtain the attentional key output; and
wherein a first model instance layer of a plurality of model instance layers of a hierarchical processing structure comprises the first instance and the second instance of the machine-learned LFM.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein a quantity of instances of the machine-learned LFM included in the first model instance layer is equal to a quantity of instances of the machine-learned LFM included in the second model instance layer of the hierarchical processing structure.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein generating the task output based on the attentional value output obtained for each of the subset of portions comprises:
for each portion of the subset of portions of the content item:
identifying, by the computing system, one or more target instances of the machine-learned LFM from a second model instance layer of the hierarchical processing structure; and
processing, by the computing system, at least the attentional value output obtained for the portion of the content item with at least one of the one or more target instances of the machine-learned LFM to obtain one or more respective second attentional value outputs.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein identifying the one or more target instances of the machine-learned LFM from the second model instance layer of the hierarchical processing structure comprises:
based on the attentional weight for a first portion of the subset of portions, identifying, by the computing system, a first target instance of the machine-learned LFM from a second model instance layer of the hierarchical processing structure for the attentional weight for the first portion of the subset of portions; and
based on the attentional weight for a second portion of the subset of portions, identifying, by the computing system, a second target instance of the machine-learned LFM from the second model instance layer of the hierarchical processing structure for the attentional weight for the second portion of the subset of portions.

### Claim 6 (depends on 5)

The computing system of claim 5, wherein the first target instance and the second target instance of the machine-learned LFM both comprise a same instance of the machine-learned LFM, and wherein processing the attentional value output obtained for the portion of the content item with each of the one or more target instances of the machine-learned LFM to obtain the one or more respective second attentional value outputs comprises:
merging, by the computing system, the attentional value outputs for the first portion and the second portion of the subset of portions with the first target instance of the machine-learned LFM to obtain a merged attentional value output; and
wherein the task output is generated based at least in part on the merged attentional value output.

### Claim 7 (depends on 5)

The computing system of claim 5, wherein the first target instance and the second target instance of the machine-learned LFM both comprise different instances of the machine-learned LFM, and wherein processing the attentional value output obtained for the portion of the content item with each of the one or more target instances of the machine-learned LFM to obtain the one or more respective second attentional value outputs comprises:
processing, by the computing system, the portion of the content item with a machine-learned LFM) to obtain an attentional value output comprising a summarization of the portion of the content item
merging, by the computing system, the attentional value outputs for the first portion and the second portion of the subset of portions with the first target instance of the machine-learned LFM to obtain a merged attentional value output; and
wherein the task output is generated based at least in part on the third attentional value output.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein processing the attentional value output with the machine-learned LFM to obtain the attentional key output comprises:
processing, by the computing system, the summarization of the portion of the content item from the attentional value output and an attentional key prompt to obtain the attentional key output, wherein the attentional key prompt is descriptive of instructions to identify key words and/or phrases from the portion of the content item.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein processing the attentional value output with the machine-learned LFM to obtain the attentional query output comprises:
processing, by the computing system, the summarization of the portion of the content item from the attentional value output and an attentional query prompt to obtain the attentional query output, wherein the attentional query prompt is descriptive of instructions to identify thematic elements associated with the portion of the content item from the portion of the content item.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein processing the portion of the content item with the machine-learned LFM to obtain the attentional value output comprises:
processing, by the computing system, the portion of the content item and an attentional value prompt with the machine-learned LFM to obtain the attentional value output, wherein the attentional value prompt is descriptive of instructions to summarize the portion of the content item.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the content item comprises at least one of:
video content;
image content;
audio content;
Mixed Reality (MR) content; or
textual content.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein the content item comprises video content;
wherein processing the portion of the content item with the machine-learned LFM to obtain the attentional value output comprises processing, by the computing system, a portion of the video content with the machine-learned LFM to obtain the attentional value output comprising a summarization of one or more scenes depicted by the portion of the video content;
wherein processing the attentional value output with the machine-learned LFM to obtain the attentional query output comprises processing, by the computing system, the attentional value output with the machine-learned LFM to obtain the attentional query output descriptive of thematic elements associated with the portion of the video content; and
wherein processing the attentional value output with the machine-learned LFM to obtain the attentional key output comprises processing, by the computing system, the attentional value output with the machine-learned LFM to obtain the attentional key output comprising key words and/or phrases spoken during the one or more scenes depicted by the portion of the video content.

### Claim 13 (depends on 11)

The computer-implemented method of claim 11, wherein the content item further comprises audio content synchronized with the video content, and wherein processing the portion of the video content with the machine-learned LFM to obtain the attentional value output comprises:
processing, by the computing system, the portion of the video content with a video encoder portion of the machine-learned LFM; and
processing, by the computing system, a portion of the audio content synchronized with the portion of the video content with an audio encoder portion of the machine-learned LFM.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, wherein processing the portion of the content item with a machine-learned LFM to obtain the attentional value output comprising the summarization of the portion of the content item comprises:
for each topic of a plurality of topics:
processing, by the computing system, the portion of the content item and a prompt indicative of the topic of the plurality of topics to obtain an attentional value sub-output of a plurality of attentional value sub-outputs, wherein the portion of the attentional value sub-output summarizes the portion of the content item based on the topic; and
aggregating, by the computing system, the attentional value sub-outputs obtained for each topic of the plurality of topics to obtain the attentional value output.

### Claim 15

A computing system, comprising:
one or more processor devices;
one or more non-transitory computer-readable media that store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
for each portion of a plurality of portions of a content item:
processing the portion of the content item with a machine-learned Large Foundational Model (LFM) to obtain an attentional value output comprising a summarization of the portion of the content item;
processing the attentional value output with the machine-learned LFM to obtain an attentional query output descriptive of thematic elements associated with the portion of the content item; and
processing the attentional value output with the machine-learned LFM to obtain an attentional key output comprising key words and/or phrases from the portion of the content item;
determining an attentional weight for each portion of the plurality of portions of the content item based on a semantic similarity between the attentional query output and the attentional key output for each of the plurality of portions;
selecting a subset of portions of the plurality of portions of the content item based on the attentional weight determined for each of the subset of portions; and
generating a task output based on the attentional value output obtained for each of the subset of portions.

### Claim 16 (depends on 15)

The computing system of claim 15, wherein processing the attentional value output with the machine-learned LFM to obtain the attentional query output comprises:
processing the attentional value output with a first instance of the machine-learned LFM to obtain the attentional query output;
wherein processing the attentional value output with the machine-learned LFM comprises:
processing the attentional value output with a second instance of the machine-learned LFM to obtain the attentional key output; and
wherein a first model instance layer of a plurality of model instance layers of a hierarchical processing structure comprises the first instance and the second instance of the machine-learned LFM.

### Claim 17 (depends on 16)

The computing system of claim 16, wherein a quantity of instances of the machine-learned LFM included in the first model instance layer is equal to a quantity of instances of the machine-learned LFM included in the second model instance layer of the hierarchical processing structure.

### Claim 18 (depends on 16)

The computing system of claim 16, wherein generating the task output based on the attentional value output obtained for each of the subset of portions comprises:
for each portion of the subset of portions of the content item:
identifying one or more target instances of the machine-learned LFM from a second model instance layer of the hierarchical processing structure; and
processing at least the attentional value output obtained for the portion of the content item with at least one of the one or more target instances of the machine-learned LFM to obtain one or more respective second attentional value outputs.

### Claim 19 (depends on 18)

The computing system of claim 18, wherein identifying the one or more target instances of the machine-learned LFM from the second model instance layer of the hierarchical processing structure comprises:
based on the attentional weight for a first portion of the subset of portions, identifying a first target instance of the machine-learned LFM from a second model instance layer of the hierarchical processing structure for the attentional weight for the first portion of the subset of portions; and
based on the attentional weight for a second portion of the subset of portions, identifying a second target instance of the machine-learned LFM from the second model instance layer of the hierarchical processing structure for the attentional weight for the second portion of the subset of portions.

### Claim 20

One or more non-transitory computer-readable media that store instructions that, when executed by one or more processor devices, cause the one or more processor devices to perform operations, the operations comprising:
for each portion of a plurality of portions of a content item:
processing the portion of the content item with a machine-learned Large Foundational Model (LFM) to obtain an attentional value output comprising a summarization of the portion of the content item;
processing the attentional value output with the machine-learned LFM to obtain an attentional query output descriptive of thematic elements associated with the portion of the content item; and
processing the attentional value output with the machine-learned LFM to obtain an attentional key output comprising key words and/or phrases from the portion of the content item;
determining an attentional weight for each portion of the plurality of portions of the content item based on a semantic similarity between the attentional query output and the attentional key output for each of the plurality of portions;
selecting a subset of portions of the plurality of portions of the content item based on the attentional weight determined for each of the subset of portions; and
generating a task output based on the attentional value output obtained for each of the subset of portions.
