# US 2019/0244080 A1 · Neural Network Processor with On-Chip Convolution Kernel Storage

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244080 A1
- Document ID: US20190244080A1
- Status: Published application
- Kind: A1
- Application number: 15/887,367
- Filed: 2018-02-02
- Published: 2019-08-08
- Earliest claimed: 2018-02-02
- Assignees: Wisconsin Alumni Research Foundation
- Applicants: Wisconsin Alumni Research Foundation
- Inventors: Jing Li; Jialiang Zhang
- Web page: https://patentgrep.com/patent/US20190244080A1

## CPC

- G06N 3/042: Knowledge-based neural networks; Logical representations of neural networks
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A neural network processor architecture provides decompression circuitry that can exploit patterns of data in kernel weights of a convolutional neural net as flattened to a vector, the compression allowing reduced kernel data storage costs including on-chip-storage.

## Claims

### Claim 1

A neural network processor operating to receive data and to classify that data comprising:
an input register holding received data for classification;
a codebook storage memory holding data permitting a mapping of individual codeword values to patterns of multiple kernel weight values related to a kernel of a neural network trained to provide a set of classifications;
a codeword memory holding codeword data to provide generation of a reconstructed kernel by indexing the codebook storage memory with codeword data; and
arithmetic circuitry communicating with the input register, the codebook storage memory, and the codeword memory to generate output representing a dot product between the received data and the reconstructed kernel to classify the received data according to the set of classifications.

### Claim 2 (depends on 1)

The neural network processor of claim 1 wherein the patterns of multiple kernel weight values are contiguous weight values within a matrix of the kernel.

### Claim 3 (depends on 2)

The neural network processor of claim 2 wherein the patterns of multiple kernel weight values are contiguous weight values along a predetermined dimension of the kernel data.

### Claim 4 (depends on 3)

The neural network processor of claim 3 wherein the patterns of multiple kernel weight values have beginning and end of values aligned within the matrix of the kernel.

### Claim 5 (depends on 4)

The neural network processor of claim 4 wherein the beginning and end values extend at least a full dimension of the matrix of the kernel.

### Claim 6 (depends on 1)

The neural network processor of claim 1 wherein the reconstructed kernel is a scalar compression of the kernel of the neural network trained to provide the set of classifications.

### Claim 7 (depends on 6)

The neural network processor of claim 6 wherein the scalar compression replaces a range of kernel data values with a predetermined scalar value.

### Claim 8 (depends on 1)

The neural network processor of claim 1 wherein the patterns of multiple kernel weight values are a product quantization of vectors of the kernel of the neural network trained to provide a set of classifications.

### Claim 9 (depends on 8)

The neural network processor of claim 8 wherein the product quantization provides multiple individual codeword values associated with different but contiguous patterns of multiple kernel weights.

### Claim 10 (depends on 1)

The neural network processor of claim 1 wherein the arithmetic circuitry employs the data of the codebook storage memory to precompute a set of multiplications between the received data and data of the codebook storage memory to populate a first lookup table and repeatedly uses the precomputed set of multiplications according to data of the codeword memory in generation of the output representing a dot product between the received data and the reconstructed kernel.

### Claim 11 (depends on 10)

The neural network processor of claim 10 wherein the arithmetic circuit further includes a second lookup table and further operates to populate the second lookup table with sums formed using the data of the codeword storage memory and the first lookup table and uses the second lookup table to form the dot product.

### Claim 12 (depends on 11)

The neural network processor of claim 11 wherein the codebook storage memory holds a scalar-quantized codebook that can be reconstructed into a product-quantized codebook using scalar-quantized codewords of the codeword memory, and wherein the product-quantized codebook can be reconstructed into a kernel using product-quantized codewords stored in the codeword memory.

### Claim 13 (depends on 11)

The neural network processor of claim 11 wherein the data of the first and second lookup tables are stored for reuse between successive received data separated by a convolution of the kernel on an input data set.

### Claim 14 (depends on 1)

The neural network processor of claim 1 wherein the input register, codebook storage memory, codeword memory, and arithmetic circuit are held on a single integrated circuit substrate.

### Claim 15

A method of operating a neural network processor receiving data to classify that data and having:
an input register holding received data for classification;
a codebook storage memory holding data permitting a mapping of individual codeword values to patterns of multiple kernel weight values related to a kernel of a neural network trained to provide a set of classifications;
a codeword memory holding codeword data to permit this generation of a reconstructed kernel by indexing the codebook storage memory data with the codeword data; and
arithmetic circuitry communicating with the input register, the codebook storage memory, and the codeword memory to generate output representing a dot product between the received data and the reconstructed kernel to classify the received data according to the set of classifications; the method comprising
(a) training a neural network having weight values;
(b) decomposing the weight values into vectors and compressing the vectors to create a codebook mapping individual codeword values to patterns of multiple kernel weight values and corresponding codeword data allowing reconstruction of the weight values;
(c) loading the codebook and the codeword data into the codebook storage memory and codeword memory respectively;
(d) operating the neural network processor to receive data for classification and to generate data representing a dot product between the received data and a kernel reconstructed from data of the codebook storage memory and codeword memory to classify the received data according to the set of classifications.
