Generating an output for a neural network output layer
AssigneeGoogle Inc.
InventorReginald Clifford Young
Life of this patent
Estimating the term…
Loading patent…
AssigneeGoogle Inc.
InventorReginald Clifford Young
Estimating the term…
Systems, methods, and apparatus, including computer programs encoded on a computer storage medium for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer. In one aspect, the methods include obtaining a layer output generated by the one or more initial neural network layers and processing the layer output through the softmax output layer to generate a neural network output. Processing the layer output through the softmax output layer includes determining, for each possible output value, a number of occurrences in the layer output values; for each possible output value occurring in the layer output values, determining a respective exponentiation measure; determining a normalization factor for the layer output by combining the exponentiation measures in accordance with the number of occurrences of the possible output values; and determining, for each of layer output values, a softmax probability value.
A method of processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the method comprising:
A system comprising one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the operations comprising:
A non-transitory computer storage medium encoded with instructions that, when executed by one or more computers, cause the one or more computers to perform operations for processing a network input through a neural network having one or more initial neural network layers followed by a softmax output layer to generate a neural network output for the network input, wherein the initial neural network layers are implemented by a processing system that performs computations specified by the one or more initial neural network layers using quantized arithmetic such that output values generated by the processing system can take only values from a predetermined finite set of values, the operations comprising:
Dependencies are inferred from the claim text.