Using embedding functions with a deep network
AssigneeGoogle Inc.
InventorsGregory S. Corrado, Kai Chen, Jeffrey A. Dean, Gary R. Holt, Julian P. Grady, Sharat Chikkerur, David W. Sculley, II
Life of this patent
Estimating the term…
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using embedding functions with a deep network. An input that includes features of multiple different feature types can be processed using respective embedding functions for each of the feature types. The outputs of these respective embedding functions can then be processed using a deep neural network.
Claims20
A method performed by one or more computers, the method comprising:
- receiving a system input comprising a plurality of features, wherein two or more of the features are of a different feature type;
- processing each of the features using a respective embedding function to generate a respective numeric representation of the feature that comprises one or more respective numeric values, wherein each of the embedding functions operates independently of each of the other embedding functions, and wherein each of the embedding functions is specific to features of a respective feature type;
- processing an input comprising the respective numeric representations of the features using a deep network to generate a first alternative representation of the input, wherein the deep network is a machine learning model composed of a plurality of levels of non-linear operations;
- and
- processing the first alternative representation of the system input using an output layer to generate a prediction for the system input.
A system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
- receiving a system input comprising a plurality of features, wherein two or more of the features are of a different feature type;
- processing each of the features using a respective embedding function to generate a respective numeric representation of the feature that comprises one or more respective numeric values, wherein each of the embedding functions operates independently of each of the other embedding functions, and wherein each of the embedding functions is specific to features of a respective feature type;
- processing an input comprising the respective numeric representations of the features using a deep network to generate a first alternative representation of the input, wherein the deep network is a machine learning model composed of a plurality of levels of non-linear operations;
- and
- processing the first alternative representation of the system input using an output layer to generate a prediction for the system input.
One or more non-transitory computer readable media storing instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:
- receiving a system input comprising a plurality of features, wherein two or more of the features are of a different feature type;
- processing each of the features using a respective embedding function to generate a respective numeric representation of the feature that comprises one or more respective numeric values, wherein each of the embedding functions operates independently of each of the other embedding functions, and wherein each of the embedding functions is specific to features of a respective feature type;
- processing an input comprising the respective numeric representations of the features using a deep network to generate a first alternative representation of the input, wherein the deep network is a machine learning model composed of a plurality of levels of non-linear operations;
- and
- processing the first alternative representation of the system input using an output layer to generate a prediction for the system input.
Dependencies are inferred from the claim text.