US 2019/0294413 A1Published application

Accelerated quantized multiply-and-add operations

AssigneeAmazon Technologies, Inc.

InventorsDana Michelle Vantrease, Randy Huang, Ron Diamant, Thomas Elmer, Sundeep Amirineni

Life of this patent

Estimating the term…

Loading patent…

Abstract

Disclosed herein are techniques for accelerating convolution operations or other matrix multiplications in applications such as neural network. A computer-implemented method includes receiving low-precision inputs for a convolution operation from a storage device, and subtracting a low-precision value representing a high-precision zero value from the low-precision inputs to generate difference values, where the low-precision inputs are asymmetrically quantized from high-precision inputs. The method also includes performing multiplication and summation operations on the difference values to generate a sum of products, and generating a high-precision output by scaling the sum of products with a scaling factor.

Claims20

Dependent claims

A computer-implemented method, comprising:

receiving floating point inputs to a convolution layer of an artificial neural network;
quantizing the floating point inputs linearly into n-bit integer inputs, wherein:
n is a multiple of 8;
a minimum value of the floating point inputs is quantized as integer zero;
a maximum value of the floating point inputs is quantized as 2n−1;
and
a zero value of the floating point inputs is quantized as an integer representing the zero value of the floating point inputs with no quantization error;
storing the integer inputs in a memory;
reading, by the convolution layer, the integer inputs from the memory;
creating difference values by subtracting, by a subtraction engine, the integer representing the zero value of the floating point inputs from the integer inputs such that the zero value of the floating point inputs is represented by an integer value zero;
performing, by the convolution layer, one set of (n+1)-bit integer multiplication and summation operations on the difference values to generate a sum of products;
and
generating, by an activation layer, a pooling layer, or an output buffer of the artificial neural network, a floating point output by scaling the sum of products with a floating point scaling factor.

10 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/934,681Filed 23 Mar 2018
Published: 26 Sep 2019
Term ends: Not estimatedThe source records don’t support an estimate.

Classification12 codes

G06F 7/5443
For evaluating functions by calculation: sum of products
G06F 7/16
Merging, i.e. combining at least two sets of record carriers each arranged in the same ordered sequence to produce a single set having the same…:…
G06F 7/5095
For multiple operands, e.g. digital integrators: word-serial, i.e. with an accumulator-register
G06F 17/15
Complex mathematical operations: correlation function computation including computation of convolution operations