# US 2025/0384346 A1 · Generation and Processing of Reduced Dimensionality Embeddings

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0384346 A1
- Document ID: US20250384346A1
- Status: Published application
- Kind: A1
- Application number: 19/238,222
- Filed: 2025-06-13
- Published: 2025-12-18
- Earliest claimed: 2024-06-14
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Jinsung Yoon; Rajarishi Sukant Sinha; Sercan Omer Arik
- Web page: https://patentgrep.com/patent/US20250384346A1

## CPC

- G06N 20/00: Machine learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, systems, devices, and non-transitory computer readable media for generating reduced dimensionality embeddings are provided. The disclosed technology can include receiving high-dimensionality embeddings comprising high-dimensionality vectors comprising a first plurality of dimensions. Based on inputting the high-dimensionality embeddings into a dimensionality reduction model that is configured to reduce the dimensionality of vectors of embeddings, a plurality of low-dimensionality embeddings comprising a plurality of low-dimensionality vectors can be generated. Each of the plurality of low-dimensionality vectors can be based on the high-dimensionality vectors of the high-dimensionality embeddings and can comprise a second plurality of dimensions that is smaller than the first plurality of dimensions of the high-dimensionality vectors. The plurality of low-dimensionality embeddings can be stored.

## Claims

### Claim 1

A computer-implemented method of reducing a dimensionality of embeddings, the computer-implemented method comprising:
receiving, by a computing system comprising one or more processors, high-dimensionality embeddings comprising high-dimensionality vectors comprising a first plurality of dimensions;
generating, by the computing system, based on inputting the high-dimensionality embeddings into a dimensionality reduction model that is configured to reduce the dimensionality of vectors of embeddings, a plurality of low-dimensionality embeddings comprising a plurality of low-dimensionality vectors, wherein each of the plurality of low-dimensionality vectors is based on the high-dimensionality vectors of the high-dimensionality embeddings and comprises a second plurality of dimensions that is smaller than the first plurality of dimensions of the high-dimensionality vectors; and
storing, by the computing system, the plurality of low-dimensionality embeddings.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, by the computing system, high-dimensionality training embeddings comprising high-dimensionality training vectors comprising a third plurality of dimensions;
generating, by the computing system, based on inputting the high-dimensionality training embeddings into the dimensionality reduction model, low-dimensionality training embeddings comprising low-dimensionality training vectors comprising a fourth plurality of dimensions that is smaller than the third plurality of dimensions of the high-dimensionality training embeddings;
determining, by the computing system, an amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings;
determining, by the computing system, a loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings; and
modifying, by the computing system, based on the loss, a weighting of parameters of the dimensionality reduction model, wherein the weighting of the parameters is modified to minimize the loss.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the determining, by the computing system, an amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings comprises:
determining, by the computing system, a cosine similarity between the high-dimensionality embeddings and the low-dimensionality embeddings.

### Claim 4 (depends on 2)

The computer-implemented method of claim 2, wherein the determining, by the computing system, a loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings, a loss comprises:
determining, by the computing system, a top-k similarity loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings.

### Claim 5 (depends on 2)

The computer-implemented method of claim 2, wherein the determining, by the computing system, a loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings, a loss comprises:
determining, by the computing system, a pairwise loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings.

### Claim 6 (depends on 2)

The computer-implemented method of claim 2, wherein the determining, by the computing system, a loss based on the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings, a loss comprises:
comparing, by the computing system, the high-dimensionality training vectors to the low-dimensionality training vectors; and
determining, by the computing system, the amount of similarity based on the comparison of the high-dimensionality training vectors to the low-dimensionality training vectors.

### Claim 7 (depends on 2)

The computer-implemented method of claim 2, wherein the loss is positively correlated with the amount of similarity between the high-dimensionality training embeddings and the low-dimensionality training embeddings at a plurality of different fourth pluralities of dimensions that are smaller than the third plurality of dimensions of the high-dimensionality training embeddings.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, by the computing system, training input comprising high-dimensionality training corpus embeddings, high-dimensionality training query embeddings, and high-dimensionality query-corpus pairs;
generating, by the computing system, based on inputting the training input into the dimensionality reduction model, training output comprising low-dimensionality training corpus embeddings and low-dimensionality training query embeddings, wherein a dimensionality of the low-dimensionality training corpus embeddings is lower than a dimensionality of the high-dimensionality training corpus embeddings, and wherein a dimensionality of the low-dimensionality training query embeddings is lower than a dimensionality of the high-dimensionality training query embeddings;
determining, by the computing system, an amount of similarity between the training input and the training output;
determining, by the computing system, a loss based on the amount of similarity between the training input and the training output; and
modifying, by the computing system, based on the loss, a weighting of parameters of the dimensionality reduction model, wherein the weighting of the parameters is modified to minimize the loss.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the determining, by the computing system, an amount of similarity between the training input and the training output comprises:
determining, by the computing system, a cosine similarity between the high-dimensionality training corpus embeddings and the low-dimensionality training corpus embeddings; and
determining, by the computing system, a cosine similarity between the high-dimensionality training query embeddings and the low-dimensionality training query embeddings.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the determining, by the computing system, a loss based on the amount of similarity between the training input and the training output comprises:
determining, by the computing system, a ranking loss based on the amount of similarity between the high-dimensionality training corpus embeddings and the low-dimensionality training corpus embeddings; and
determining, by the computing system, the ranking loss based on the amount of similarity between the high-dimensionality training query embeddings and the low-dimensionality training query embeddings.

### Claim 11 (depends on 8)

The computer-implemented method of claim 8, wherein the loss is positively correlated with the amount of similarity between the training input and the training output at a plurality of different dimensionalities in which the dimensionality of the training output is smaller than the dimensionality of the training input.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, by the computing system, a plurality of high-dimensionality training embeddings comprising a plurality of high-dimensionality training vectors comprising a fifth plurality of dimensions of different sizes;
generating, by the computing system, based on inputting the high-dimensionality training embeddings into the dimensionality reduction model, a plurality of adapted dimensionality training embeddings corresponding to the plurality of high-dimensionality training embeddings and comprising a plurality of adapted dimensionality training vectors that are equal in size to each of the plurality of high-dimensionality training vectors respectively;
determining, by the computing system, an amount of similarity between the high-dimensionality training embeddings and the adapted dimensionality training embeddings;
determining, by the computing system, a loss based on the amount of similarity between the high-dimensionality training embeddings and the adapted dimensionality training embeddings; and
modifying, by the computing system, based on the loss, a weighting of parameters of the dimensionality reduction model, wherein the weighting of the parameters is modified to minimize the loss.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein the dimensionality reduction model comprises a multilayer perceptron (MLP), wherein the plurality of low-dimensionality embeddings comprise Matryoshka embeddings, and wherein the high-dimensionality embeddings and the plurality of low-dimensionality embeddings comprise a numerical representation of one or more images, one or more video segments, one or more text segments, or one or more audio segments.

### Claim 14 (depends on 1)

The computer-implemented method of claim 1, wherein the dimensionality reduction model is trained based on unsupervised learning operations comprising determining a top-k similarity loss or a pairwise similarity loss, and wherein determining the top-k similarity loss or the pairwise similarity loss comprises comparing document embeddings to other document embeddings.

### Claim 15 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of low-dimensionality embeddings comprise two or more low-dimensionality embeddings that have a lower dimensionality than the high-dimensionality embeddings.

### Claim 16 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of low-dimensionality embeddings have a plurality of different vectors that have a plurality of different dimensions.

### Claim 17 (depends on 1)

The computer-implemented method of claim 1, wherein the dimensionality reduction model is trained based on determining a ranking loss, and wherein determining the ranking loss comprises comparing corpus embeddings to other corpus embeddings, comparing query embeddings to other query embeddings, or comparing query-corpus pairs to other query corpus pairs.

### Claim 18 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving, by the computing system, a query based on the high-dimensionality embeddings;
determining, by the computing system, based on the query, a low-dimensionality embedding of the plurality of low-dimensionality embeddings that corresponds to the query; and
accessing, by the computing system, a low-dimensionality embedding of the plurality of low-dimensionality embeddings.

### Claim 19

One or more tangible non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:
receiving a high-dimensionality embeddings comprising high-dimensionality vectors comprising a first plurality of dimensions;
generating, based on inputting the high-dimensionality embeddings into a dimensionality reduction model that is configured to reduce the dimensionality of vectors of embeddings, a plurality of low-dimensionality embeddings comprising a plurality of low-dimensionality vectors, wherein each of the plurality of low-dimensionality vectors is based on the high-dimensionality vectors of the high-dimensionality embeddings and comprises a second plurality of dimensions that is smaller than the first plurality of dimensions of the high-dimensionality vectors; and
storing the plurality of low-dimensionality embeddings.

### Claim 20

A computing system comprising:
one or more processors;
one or more non-transitory computer-readable media storing instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
receiving high-dimensionality embeddings comprising high-dimensionality vectors comprising a first plurality of dimensions;
generating, based on inputting the high-dimensionality embeddings into a dimensionality reduction model that is configured to reduce the dimensionality of vectors of embeddings, a plurality of low-dimensionality embeddings comprising a plurality of low-dimensionality vectors, wherein each of the plurality of low-dimensionality vectors is based on the high-dimensionality vectors of the high-dimensionality embeddings and comprises a second plurality of dimensions that is smaller than the first plurality of dimensions of the high-dimensionality vectors; and
storing the plurality of low-dimensionality embeddings.
