Large-scale classification in neural networks using hashing

A method for processing an input through each of a plurality of layers of a neural network to generate an output, wherein each of the plurality of layers of the neural network comprises a respective plurality of nodes, the method comprising, for a particular layer of the plurality of layers:

• determining, by a classification system, one or more hash codes of an activation vector that is input for the particular layer;
• selecting one or more nodes in the particular layer using the one or more hash codes of the activation vector as input to a lookup function for a hash table that maps hash codes of activation vectors to data for weight values for nodes in the particular layer;
• and
• generating an output for the particular layer by combining the weight values for the selected nodes with the activation vector.

A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:

• determining one or more hash codes of an activation vector that is input for a particular layer of a neural network;
• selecting one or more nodes in the particular layer using the one or more hash codes of the activation vector as input to a lookup function for a hash table that maps hash codes of activation vectors to data for weight values for nodes in the particular layer;
• and
• generating an output for the particular layer by combining the weight values for the selected nodes with the activation vector.

A computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations comprising:

• determining one or more hash codes of an activation vector that is input for a particular layer of a neural network;
• selecting one or more nodes in the particular layer using the one or more hash codes of the activation vector as input to a lookup function for a hash table that maps hash codes of activation vectors to data for weight values for nodes in the particular layer;
• and
• generating an output for the particular layer by combining the weight values for the selected nodes with the activation vector.