# US 2025/0061310 A1 · Methods and systems for executing a neural network with bit addition-based inner product operator

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061310 A1
- Document ID: US20250061310A1
- Status: Published application
- Kind: A1
- Application number: 18/450,142
- Filed: 2023-08-15
- Published: 2025-02-20
- Earliest claimed: 2023-08-15
- Inventors: Xinlin LI; Vanessa COURVILLE; Vahid PARTOVI NIA; Boxing CHEN
- Web page: https://patentgrep.com/patent/US20250061310A1

## CPC

- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/045: Combinations of networks
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and methods for computing a neural network layer of a neural network are described. A float shift inner product operator is described for computing an inner product between a floating point input vector and a weight vector having weight values that are power of two values. An inner product is computed by performing addition on the sign bits and exponent bits of a floating point input element and corresponding weight element.

## Claims

### Claim 1

A computing system for computing an output of a neural network layer of a neural network, the computing system comprising:
a memory storing a weight vector for the neural network layer, each element of the weight vector being a weight element encoded as a bit string including a sign bit representing a sign value of the weight element and a set of exponent bits representing an exponent of a weight value; and
a processing unit coupled to the memory, the processing unit comprising:
circuitry configured to receive a floating point input vector to the neural network layer and the weight vector for the neural network layer, each element of the floating point input vector being a floating point input element encoded as a floating point bit string including a sign bit representing a sign value of the floating point input element and a set of exponent bits representing an exponent of the floating point input element;
circuitry configured to compute an inner product between the floating point input vector and the weight vector by:
for each floating point input element in the floating point input vector and a corresponding weight element in the weight vector, performing addition to add the sign bit of the weight element to the sign bit of the floating point input element and to add the set of exponent bits of the weight element to the set of exponent bits of the floating point input element, to generate a respective weighted input element; and
performing floating point summation of the respective weighted input elements to generate the inner product; and
circuitry configured to output the inner product as an output element of an output vector of the neural network layer.

### Claim 2 (depends on 1)

The computing system of claim 1, wherein each weight element is encoded as a shift bit value including the sign value of the weight element and the weight value that is a power of two value or zero.

### Claim 3 (depends on 1)

The computing system of claim 1, wherein each weight element is encoded as a dense shift bit value including the sign value of the weight element and the weight value that is a non-zero power of two value.

### Claim 4 (depends on 1)

The computing system of claim 1, wherein the weight vector is stored as a low-bit encoded weight vector having low-bit encoded weight elements, each low-bit encoded weight element representing a corresponding weight element using fewer bits, and the low-bit encoded weight vector is converted to the weight vector.

### Claim 5 (depends on 1)

The computing system of claim 1, wherein each weight element has a bit-length equal to a bit-length of the corresponding floating point input element.

### Claim 6 (depends on 1)

The computing system of claim 1, wherein each weight element has a bit-length equal to a bit-length of the sign bit plus the set of exponent bits of the corresponding floating point input element.

### Claim 7 (depends on 1)

The computing system of claim 1, wherein the circuitry configured to compute the inner product between the floating point input vector and the weight vector includes circuitry for an integer addition operator for performing the addition.

### Claim 8 (depends on 1)

The computing system of claim 1, wherein the circuitry configured to compute the inner product between the floating point input vector and the weight vector includes circuitry for a binary addition operator for performing the addition.

### Claim 9 (depends on 1)

The computing system of claim 1, wherein the neural network layer is a fully connected neural network layer, and wherein the weight vector represents a multi-dimensional weight tensor.

### Claim 10 (depends on 1)

The computing system of claim 1, wherein the neural network layer is a self-attention neural network layer, and wherein the weight vector represents a weights of at least one of a query, key or value matrix.

### Claim 11 (depends on 1)

The computing system of claim 1, wherein the neural network layer is a convolutional neural network layer, wherein the weight vector represents a convolutional kernel.

### Claim 12 (depends on 1)

The computing system of claim 1, wherein the processing unit is a dedicated neural network accelerator.

### Claim 13 (depends on 1)

The computing system of claim 1, wherein the processing unit further comprises:
circuitry configured to convert each weight element of the weight vector by rounding the weight value of each weight element to a power of two value;
wherein the weight vector of converted weight elements is used to compute the inner product.

### Claim 14

A method for computing an output of a neural network layer of a neural network, the method comprising:
receiving a floating point input vector, each element of the floating point input vector being a floating point input element encoded as a floating point bit string including a sign bit representing a sign value of the floating point input element and a set of exponent bits representing an exponent of the floating point input element;
obtaining a weight vector for the neural network layer, each element of the weight vector being a weight element encoded as a bit string including a sign bit representing a sign value of the weight element and a set of exponent bits representing an exponent of a weight value;
computing an inner product between the floating point input vector and the weight vector by:
for each floating point input element in the floating point input vector and a corresponding weight element in the weight vector, performing addition to add the sign bit of the weight element to the sign bit of the floating point input element and to add the set of exponent bits of the weight element to the set of exponent bits of the floating point input element, to generate a respective weighted input element; and
performing floating point summation of the respective weighted input elements to generate the inner product; and
outputting the inner product as an output element of an output vector of the neural network layer.

### Claim 15 (depends on 14)

The method of claim 14, wherein each weight element is encoded as a shift bit value including the sign value of the weight element and the weight value that is a power of two value or zero.

### Claim 16 (depends on 14)

The method of claim 14, wherein each weight element is encoded as a dense shift bit value including the sign value of the weight element and the weight value that is a non-zero power of two value.

### Claim 17 (depends on 14)

The method of claim 14, wherein obtaining the weight vector comprises converting a low-bit encoded weight vector having low-bit encoded weight elements, each low-bit encoded weight element representing a corresponding weight element using fewer bits, to the weight vector.

### Claim 18 (depends on 14)

The method of claim 14, wherein each weight element has a bit-length equal to a bit-length of the corresponding floating point input element.

### Claim 19 (depends on 14)

The method of claim 14, wherein each weight element has a bit-length equal to a bit-length of the sign bit plus the set of exponent bits of the corresponding floating point input element.

### Claim 20 (depends on 14)

The method of claim 14, wherein performing the addition comprises performing an integer addition operation.
