# US 2025/0165469 A1 · Machine Learning Models as a Differentiable Search Index for Directly Predicting Resource Retrieval Results

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0165469 A1
- Document ID: US20250165469A1
- Status: Published application
- Kind: A1
- Application number: 18/837,122
- Filed: 2023-02-09
- Published: 2025-05-22
- Earliest claimed: 2022-02-09
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Yi Tay; Vinh Quoc Tran; William Weston Cohen; Donald Arthur Metzler, JR.
- Web page: https://patentgrep.com/patent/US20250165469A1

## CPC

- G06F 16/24542: {Plan optimisation}
- G06F 16/316: {Indexing structures}
- G06F 16/901: Indexing; Data structures therefor; Storage structures (for retrieval from the web G06F16/951)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Provided are systems and methods for training and/or use of a machine learning model that can directly predict one or more resources that are responsive to a query as an output of the model. In particular, the present disclosure demonstrates that information retrieval can be accomplished with a single machine learning model (e.g., that has a neural network architecture such as, for example, a Transformer architecture) in which all information about the corpus is encoded in the parameters of the model. To this end, the present disclosure introduces the Differentiable Search Index (DSI), a new paradigm that learns a query-to-result (e.g., in text-to-text format) model that will map queries (e.g., text strings) directly to relevant resource identifiers (“docids”) (e.g., text and/or number strings that identify relevant resources); in other words, a DSI model answers queries directly using only its parameters, dramatically simplifying retrieval

## Claims

### Claim 1

A computer-implemented method to perform resource retrieval with improved computational efficiency, the method comprising:
obtaining, by a computing system comprising one or more computing devices, a query;
processing, by the computing system, the query with a machine-learned resource retrieval model to generate a model prediction from the machine-learned resource retrieval model,
wherein the model prediction directly predicts one or more resources that are predicted to be responsive to the query from a resource corpus containing a plurality of resources,
wherein a plurality of resource identifiers are respectively associated with the plurality of resources, and
wherein the model prediction comprises the resource identifiers for the one or more resources that are predicted to be responsive to the query;
providing, by the computing system, the model prediction as an output.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
the machine-learned resource retrieval model comprises a sequence-to-sequence model that receives and processes the query as an input sequence to generate one or more predicted output sequences as the model prediction; and
the one or more predicted output sequences correspond to the resource identifiers of the one or more resources that are predicted to be responsive to the query.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the respective resource identifier associated with each resource in the plurality of resources comprises an unstructured atomic identifier.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the respective resource identifier associated with each resource in the plurality of resources comprises an unstructured string identifier.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the respective resource identifier associated with each resource in the plurality of resources comprises a structured semantic identifier.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the respective structured semantic identifier associated with each resource in the plurality of resources has been generated via iterative clustering of a plurality of embeddings respectively associated with the plurality of resources.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the machine-learned resource retrieval model has been trained using an indexing loss function, wherein the indexing loss function evaluates an ability of the machine-learned resource retrieval model to output the resource identifier associated with a particular resource when provided with data descriptive of the particular resource as an input.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the data descriptive of the particular resource comprises:
direct indexing tokens extracted from the particular resource;
set indexing tokens extracted from the particular resource; or
inverted indexing tokens extracted from the particular resource.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the machine-learned resource retrieval model has been trained using a retrieval loss function, wherein the retrieval loss function evaluates an ability of the machine-learned resource retrieval model to output the resource identifier associated with a particular resource when provided with a training query for which the particular resource has been labeled as a response.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the machine-learned resource retrieval model has been trained using an indexing loss function and a retrieval loss function in a multi-task training approach.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the model prediction from the machine-learned resource retrieval model comprises a softmax output over the plurality of resources.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, wherein the model prediction from the machine-learned resource retrieval model comprises one or more beam search results generated by performance of a sequential beam search.

### Claim 13

A computing system for training a model to perform resource retrieval with improved computational efficiency, the computing system comprising:
one or more processors; and
one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations comprising:
obtaining, by the computing system, a resource corpus comprising a plurality of resources, wherein a respective resource identifier is associated with each of the plurality of resources; and
for each of one or more input resources of the plurality of resources:
processing, by the computing system, data descriptive of the input resource with a resource retrieval model to generate a predicted resource identifier for the input resource;
evaluating an indexing loss function that compares the predicted resource identifier to the actual resource identifier for the input resource; and
modifying one or more parameters of the resource retrieval model based on the indexing loss function.

### Claim 14 (depends on 13)

The computing system of claim 13, wherein the data descriptive of the input resource comprises:
direct indexing tokens extracted from the input resource;
set indexing tokens extracted from the input resource; or
inverted indexing tokens extracted from the input resource.

### Claim 15 (depends on 13)

The computing system of claim 13, wherein the respective resource identifier associated with each resource in the plurality of resources comprises a structured semantic identifier.

### Claim 16 (depends on 15)

The computing system of claim 15, wherein the respective structured semantic identifier associated with each resource in the plurality of resources has been generated via iterative clustering of a plurality of embeddings respectively associated with the plurality of resources.

### Claim 17 (depends on 13)

The computing system of claim 13, wherein the operations further comprise:
training the resource retrieval model using a retrieval loss function, wherein the retrieval loss function evaluates an ability of the resource retrieval model to output the resource identifier associated with a particular resource when provided with a training query for which the particular resource has been labeled as a response.

### Claim 18 (depends on 17)

The computing system of claim 17, wherein the operations comprise training the resource retrieval model using the indexing loss function and the retrieval loss function in a multi-task training approach.

### Claim 19 (depends on 13)

The computing system of claim 13, wherein the model prediction from the resource retrieval model comprises a softmax output over the plurality of resources.

### Claim 20 (depends on 13)

The computing system of claim 13, wherein the model prediction from the resource retrieval model comprises one or more beam search results generated by performance of a sequential beam search.
