Back to results
US 12,681,956 B2Granted
Input data item classification using memory data item embeddings
AssigneeGoogle Inc.
InventorsAhmet Iscen, Alireza Fathi, Cordelia Luise Schmid
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a classification task on a data item. In particular, a system classifies an input data item using key and value embeddings of memory data items.
Claims30
A method performed by one or more computers, the method comprising:
- maintaining, for each of a plurality of memory data items, (i) a respective key embedding that has been generated by processing the memory data item using a first embedding neural network and (ii) a respective value embedding that has been generated by processing at least one of the memory data item or a corresponding data item associated with the memory data item using a second embedding neural network;
- receiving a query data item;
- and
- classifying the query data item using a semi-parametric model that uses the respective key and value embeddings for the plurality of memory data items, comprising:
- processing the query data item using the first embedding neural network to generate a query embedding of the query data item;
- searching through the respective key embeddings of the memory data items by using the query embedding as a query to identify a subset of memory data item;
- generating, from the query embedding and the respective value embeddings for the identified subset of memory data items that were identified from searching through the respective key embeddings of the memory data items by using the query embedding as the query, an input to a classifier neural network, wherein the input comprises a set of one or more embeddings generated by fusing the query embedding and the respective value embeddings for the identified subset of memory data items, wherein the fusing comprises (i) generating the input based on the query embedding and a combined value embedding generated from the respective value embeddings, or (ii) processing the query embedding and the respective value embeddings through a sequence of one or more attention layers;
- and
- processing the input using the classifier neural network to generate a classification output for the query data item that specifies one or more categories to which the query data item belongs.
6 more elements
A system comprising:
- one or more computers;
- and
- one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
- maintaining, for each of a plurality of memory data items, (i) a respective key embedding that has been generated by processing the memory data item using a first embedding neural network and (ii) a respective value embedding that has been generated by processing at least one of the memory data item or a corresponding data item associated with the memory data item using a second embedding neural network;
- receiving a query data item;
- and
- classifying the query data item using a semi-parametric model that uses the respective key and value embeddings for the plurality of memory data items, comprising:
- processing the query data item using the first embedding neural network to generate a query embedding of the query data item;
- searching through the respective key embeddings of the memory data items by using the query embedding as a query to identify a subset of memory data item;
- generating, from the query embedding and the respective value embeddings for the identified subset of memory data items that were identified from searching through the respective key embeddings of the memory data items by using the query embedding as the query, an input to a classifier neural network, wherein the input comprises a set of one or more embeddings generated by fusing the query embedding and the respective value embeddings for the identified subset of memory data items, wherein the fusing comprises (i) generating the input based on the query embedding and a combined value embedding generated from the respective value embeddings, or (ii) processing the query embedding and the respective value embeddings through a sequence of one or more attention layers;
- and
- processing the input using the classifier neural network to generate a classification output for the query data item that specifies one or more categories to which the query data item belongs.
9 more elements
One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
- maintaining, for each of a plurality of memory data items, (i) a respective key embedding that has been generated by processing the memory data item using a first embedding neural network and (ii) a respective value embedding that has been generated by processing at least one of the memory data item or a corresponding data item associated with the memory data item using a second embedding neural network;
- receiving a query data item;
- and
- classifying the query data item using a semi-parametric model that uses the respective key and value embeddings for the plurality of memory data items, comprising:
- processing the query data item using the first embedding neural network to generate a query embedding of the query data item;
- searching through the respective key embeddings of the memory data items by using the query embedding as a query to identify a subset of memory data item;
- generating, from the query embedding and the respective value embeddings for the identified subset of memory data items that were identified from searching through the respective key embeddings of the memory data items by using the query embedding as the query, an input to a classifier neural network, wherein the input comprises a set of one or more embeddings generated by fusing the query embedding and the respective value embeddings for the identified subset of memory data items, wherein the fusing comprises (i) generating the input based on the query embedding and a combined value embedding generated from the respective value embeddings, or (ii) processing the query embedding and the respective value embeddings through a sequence of one or more attention layers;
- and
- processing the input using the classifier neural network to generate a classification output for the query data item that specifies one or more categories to which the query data item belongs.
6 more elements
Dependencies are inferred from the claim text.