# US 2026/0037859 A1 · Hierarchical Embedding Tree for Efficiently Caching Large Contexts

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0037859 A1
- Document ID: US20260037859A1
- Status: Published application
- Kind: A1
- Application number: 18/790,725
- Filed: 2024-07-31
- Published: 2026-02-05
- Earliest claimed: 2024-07-31
- Assignees: DeepMind Technologies Limited
- Applicants: DeepMind Technologies Limited
- Inventors: Florian Nils Hartmann; Duc-Hieu Tran
- Web page: https://patentgrep.com/patent/US20260037859A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/045: Combinations of networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An example method includes obtaining a hierarchical embedding tree that embeds context data in a plurality of embedded representations respectively associated with a plurality of nodes. In the example method, each respective embedded representation associated with each respective terminal node of a plurality of terminal nodes of the plurality of nodes is generated based on a respective chunk of the context data. In the example method, each respective embedded representation associated with each respective non-terminal node of a plurality of non-terminal nodes of the plurality of nodes is generated based on the embedded representations associated with two or more of the plurality of nodes that are child nodes of the respective non-terminal node. The example method includes generating, based on an update to a chunk of the context data, updated embedded representations respectively for a plurality of nodes that are intersected by a path from a terminal node associated with the updated chunk to an ancestor node for the terminal node. The example method includes updating the hierarchical embedding tree using the updated embedded representations. The example method includes caching the updated hierarchical embedding tree for providing inputs to a machine-learned decoder model to generate responses to queries based on the updated context data.

## Claims

### Claim 1

A computer-implemented method comprising:
obtaining a hierarchical embedding tree that embeds context data in a plurality of embedded representations respectively associated with a plurality of nodes, wherein:
each respective embedded representation associated with each respective terminal node of a plurality of terminal nodes of the plurality of nodes is generated based on a respective chunk of the context data, and
each respective embedded representation associated with each respective non-terminal node of a plurality of non-terminal nodes of the plurality of nodes is generated based on the embedded representations associated with two or more of the plurality of nodes that are child nodes of the respective non-terminal node;
generating, based on an update to a chunk of the context data, updated embedded representations respectively for a plurality of nodes that are intersected by a path from a terminal node associated with the updated chunk to an ancestor node for the terminal node;
updating the hierarchical embedding tree using the updated embedded representations; and
caching the updated hierarchical embedding tree for providing inputs to a machine-learned decoder model to generate responses to queries based on the updated context data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the hierarchical embedding tree corresponds to a rope data structure.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, comprising, for a respective query:
accessing a cached hierarchical embedding tree, the cached hierarchical embedding tree comprising the hierarchical embedding tree or the updated hierarchical embedding tree;
constructing a respective input to the machine-learned decoder model based on one or more embedded representations obtained from the cached hierarchical embedding tree; and
generating a respective response for the respective query by processing the respective input using the machine-learned decoder model.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein constructing the respective input comprises:
combining the one or more embedded representations with one or more respective query embeddings to form the respective input.

### Claim 5 (depends on 3)

The computer-implemented method of claim 3, wherein the one or more embedded representations comprise a root embedding associated with a root of the cached embedding tree.

### Claim 6 (depends on 3)

The computer-implemented method of claim 3, wherein constructing the respective input comprises:
obtaining a sequence of component embeddings based on the one or more embedded representations; and
combining the sequence of component embeddings with a sequence of respective query embeddings to form an input sequence for the respective input.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein a dimensionality of the component embeddings match a dimensionality of the respective query embeddings.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the machine-learned decoder model is a sequence processing model that performs self-attention over the input sequence to generate an output sequence.

### Claim 9 (depends on 6)

The computer-implemented method of claim 6, wherein:
the cached hierarchical embedding tree stores a respective sequence of component embeddings for each respective node;
the cached hierarchical embedding tree stores a respective sequence of component embeddings for the root node; or
obtaining the sequence of component embeddings comprises processing the one or more embedded representations using a machine-learned adapter model configured to receive one or more input embeddings.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, comprising:
obtaining the update to the chunk of the context data, wherein obtaining the update to the chunk of the context data comprises:
constructing a first input to the machine-learned decoder model based on one or more embedded representations obtained from the hierarchical embedding tree before the hierarchical embedding tree is updated;
generating a first response for a first query by processing the first input using the machine-learned decoder model; and
updating the chunk with updated data that is based on the first response.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the first query comprises an instruction to edit a portion of the context data.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, comprising:
generating a first embedded representation for a first node of the hierarchical embedding tree using a first machine-learned encoder; and
generating a second embedded representation for a second node of the hierarchical embedding tree using a second machine-learned encoder.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, wherein the first machine-learned encoder is different from the second machine-learned encoder.

### Claim 14 (depends on 13)

The computer-implemented method of claim 13, wherein:
the first node corresponds to a chunk of the context data containing data of a first data modality, and wherein the first machine-learned encoder is configured to process the first data modality; and
the second node corresponds to a chunk of the context data containing data of a second data modality, and wherein the second machine-learned encoder is configured to process the second data modality.

### Claim 15 (depends on 12)

The computer-implemented method of claim 12, wherein:
the first node is the terminal node associated with the updated chunk, and wherein the first embedded representation has a first precision; and
the second node is the ancestor node, and wherein the second embedded representation has a second precision greater than the first precision.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein the second precision being greater than the first precision comprises:
a dimensionality of the second embedded representation is higher than a dimensionality of the first embedded representation; or
a bit depth used for the second embedding is higher than a bit depth used for the first embedding.

### Claim 17 (depends on 1)

The computer-implemented method of claim 1, wherein:
updating the hierarchical embedding tree using the updated embedded representations comprises:
inserting, in the hierarchical embedding tree, a non-destructive update comprising the updated embedded representations and versioning data associated with the updated embedded representations; and
the hierarchical embedding tree is configured to facilitate retrieval of a cached embedded representation based on versioning data associated with the cached embedded representation.

### Claim 18 (depends on 1)

The computer-implemented method of claim 1, comprising:
accessing a context data tree generated by a context data processing application, wherein the context data tree comprises the plurality of nodes, and wherein the cached hierarchical embedding tree inherits the plurality of nodes from the context data tree.

### Claim 19

A computer-implemented method for training one or more machine-learned encoders for generating a hierarchical embedding tree, comprising:
generating, using a first encoder of the one or more machine-learned encoders, a first terminal embedded representation of a first chunk of training context data;
generating, using a second encoder of the one or more machine-learned encoders, a second terminal embedded representation of a second chunk of training context data;
generating, using a third encoder of the one or more machine-learned encoders, a non-terminal embedded representation that represents the first terminal embedded representation and the second terminal embedded representation;
constructing a training input based on the non-terminal embedded representation;
generating a training output using a machine-learned decoder to process the training input; and
updating at least one of the first encoder, second encoder, or third encoder based on an evaluation of the training output.

### Claim 20

A computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing:
a cached hierarchical embedding tree that embeds context data in a plurality of embedded representations respectively associated with a plurality of nodes, wherein:
each respective embedded representation associated with each respective terminal node of the cached hierarchical embedding tree is generated based on a respective chunk of the context data, and
each respective embedded representation associated with each respective non-terminal node of the cached hierarchical embedding tree is generated based on the embedded representations associated with two or more child nodes of the respective non-terminal node; and
instructions that are executable by the one or more processors to cause the computing system to perform operations comprising:
generating, based on an update to a chunk of the context data, updated embedded representations respectively for a plurality of nodes of the cached hierarchical embedding tree that are intersected by a path from a terminal node associated with the updated chunk to an ancestor node for the terminal node;
updating the cached hierarchical embedding tree using the updated embedded representations; and
providing an input based on the cached hierarchical embedding tree to a machine-learned decoder model to generate a response to a query associated with the updated context data.
