Language-Based Attention Mechanisms for Machine-Learned Models
AssigneeGoogle Inc.
InventorsRamin Mehran, Nilpa Jha
Life of this patent
Estimating the term…
Loading patent…
AssigneeGoogle Inc.
InventorsRamin Mehran, Nilpa Jha
Estimating the term…
For each portion of a content item, the portion of the content item can be processed with a machine-learned Large Foundational Model (LFM) to obtain an attentional value output comprising a summarization of the portion. The attentional value output can be processed with the machine-learned LFM to obtain an attentional query output descriptive of thematic elements associated with the portion, and an attentional key output comprising key words and/or phrases from the portion. An attentional weight can be determined for each portion based on a semantic similarity between the attentional query output and the attentional key output for each portion. A subset of portions of the content item can be selected based on the attentional weight determined for the subset. A task output can be generated based on the attentional value output obtained for each of the subset of portions.
A computer-implemented method, comprising:
A computing system, comprising:
One or more non-transitory computer-readable media that store instructions that, when executed by one or more processor devices, cause the one or more processor devices to perform operations, the operations comprising:
Dependencies are inferred from the claim text.