# US 2026/0087332 A1 · Sparse codec for neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087332 A1
- Document ID: US20260087332A1
- Status: Published application
- Kind: A1
- Application number: 18/897,242
- Filed: 2024-09-26
- Published: 2026-03-26
- Earliest claimed: 2024-09-26
- Assignees: Moffett International Co., Limited
- Applicants: Moffett International Co., Limited
- Inventors: Changxu ZHANG
- Web page: https://patentgrep.com/patent/US20260087332A1

## CPC

- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/063: using electronic means
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An advanced processor is disclosed for efficient handling of sparse tensor data. The processor comprises an input buffer configured to receive an input tensor, a gather circuit that collects high-magnitude tensor data from a dense tensor to form a condensed tensor, and a scatter circuit that distributes tensor data from a condensed tensor into a sparse uncondensed tensor based on a given mask. The processor is configured to perform sparse encoding, generating a sparse representation of the input tensor that includes: (1) the condensed tensor containing high-magnitude tensor data gathered by the gather circuit, and (2) a sparse mask indicating the position information of high-magnitude tensor data in the input tensor. Additionally, the processor performs sparse decoding by scattering the condensed tensor into the sparse uncondensed tensor using the scatter circuit and the sparse mask.

## Claims

### Claim 1

A processor, comprising:
an input buffer configured to receive an input tensor;
a gather circuit configured to gather high-magnitude tensor data from a given dense tensor into a condensed tensor; and
a scatter circuit configured to scatter tensor data from a given condensed tensor into a sparse uncondensed tensor based on a given mask;
wherein the processor is configured to:
perform sparse encoding to generate a sparse representation of the input tensor, wherein the sparse representation comprises (1) the condensed tensor generated using the gather circuit based on the high-magnitude tensor data of the input tensor and (2) a sparse mask indicating position information of high-magnitude tensor data in the input tensor; and
perform sparse decoding to scatter the condensed tensor in the sparse representation into the sparse uncondensed tensor using the scatter circuit based on the sparse mask, wherein the sparse uncondensed tensor has same dimensions as the input tensor.

### Claim 2 (depends on 1)

The processor of claim 1, wherein a memory footprint for storing the sparse representation of the input tensor is less than a memory footprint for storing the input tensor.

### Claim 3 (depends on 2)

The processor of claim 2, wherein:
the high-magnitude tensor data in the condensed format is stored using a high bit-depth to achieve high-precision storage, and
the low-magnitude tensor data in the sparse mask is stored using a reduced bit-depth to achieve low-precision storage, trading off precision for increased memory efficiency.

### Claim 4 (depends on 1)

The processor of claim 1, further comprising:
a transmitter configured to transmit the sparse representation of the input tensor to a Sparse Processing Unit (SPU) for neural network computation, or to another processor for computation or decoding.

### Claim 5 (depends on 1)

The processor of claim 1, wherein the sparse representation further comprises quantization parameters used in encoding data in the input tensor.

### Claim 6 (depends on 1)

The processor of claim 1, wherein during sparse encoding, the processor is further configured to:
sort tensor data in the input tensor to identify the high-magnitude tensor data and the low-magnitude tensor data;
generate a temporary bitmask indicating position information of the high-magnitude tensor data in the tensor;
determine a first set of quantization parameters for the high-magnitude tensor data, and a second set of quantization parameters for the low-magnitude tensor data;
generate, using the gather circuit, the condensed tensor comprising the high-magnitude tensor data quantized using the first set of quantization parameters;
generate the sparse mask representing the low-magnitude tensor data based on the temporary bitmask and the second set of quantization parameters; and
output the condensed tensor, the sparse mask, the first set of quantization parameters, and the second set of quantization parameters as the sparse representation of the tensor.

### Claim 7 (depends on 6)

The processor of claim 6, wherein to generate the condensed tensor comprising the high-magnitude tensor data quantized using the first set of quantization parameters, the processor is further configured to:
feed the input tensor and the temporary bitmask to the gather circuit to gather the high-magnitude tensor data; and
quantize the high-magnitude tensor data by applying the first set of quantization parameters to the high-magnitude tensor data.

### Claim 8 (depends on 6)

The processor of claim 6, wherein to generate the sparse mask representing the low-magnitude tensor data based on the temporary bitmask and the second set of quantization parameters, the processor is further configured to:
quantize the low-magnitude tensor data in the tensor by applying the second set of quantization parameters to the tensor to obtain a temporary tensor, wherein tensor data in the temporary tensor uses a reduced bit-depth than tensor data in the tensor; and
merge the temporary tensor with the temporary bitmask to obtain the bitmask representing the low-magnitude tensor data in the tensor.

### Claim 9 (depends on 1)

The processor of claim 1, wherein the processor further comprises a quant buffer configured to receive quantization parameters, wherein:
the quantization parameters received by the quant buffer comprise a first set of quantization parameters corresponding to the condensed tensor, and a second set of quantization parameters corresponding to the sparse mask.

### Claim 10 (depends on 9)

The processor of claim 9, wherein, during sparse decoding, the processor is further configured to:
dequantize the condensed tensor to obtain a dequantized high-magnitude tensor using the first set of quantization parameters;
dequantize the sparse mask to obtain a dequantized low-magnitude tensor using the second quantization parameters; and
feed the dequantized high-magnitude tensor and the dequantized low-magnitude tensor to the scatter circuit to obtain a restored version of the input tensor.

### Claim 11 (depends on 10)

The processor of claim 10, wherein to dequantize the condensed tensor and the sparse mask, the processor is further configured to:
increase a bit-depth of tensor data in the condensed tensor and the sparse mask, wherein the increased bit-depth is same as a bit-depth of tensor data in the input tensor.

### Claim 12 (depends on 1)

The processor of claim 1, wherein each entry in the sparse mask comprises two or more bits, with a first bit representing a sign of a corresponding tensor data, and the first bit is combined with each of subsequent bits of the two or more bits to implement multi-ternary representation of quantized tensor data.

### Claim 13

A method for sparse encoding a tensor, comprising:
sorting tensor data in the tensor to identify high-magnitude tensor data and low-magnitude tensor data;
generating a bitmask for the tensor indicating position information of the high-magnitude tensor data in the tensor;
generating, using a gather circuit, a condensed tensor comprising the high-magnitude tensor data based on the bitmask;
quantizing the condensed tensor using a set of quantization parameters;
generating a sparse mask representing the low-magnitude tensor data based on (1) the bitmask and (2) the set of quantization parameters; and
outputting the quantized condensed tensor, the sparse mask, and the set of quantization parameters as a sparse representation of the tensor.

### Claim 14 (depends on 13)

The method of claim 13, wherein the gather circuit is configured to gather high-magnitude tensor data from a given dense tensor into a condensed tensor.

### Claim 15 (depends on 13)

The method of claim 13, wherein the generating the condensed tensor representing the high-magnitude tensor data comprises:
feeding the tensor and the bitmask to the gather circuit for gathering the high-magnitude tensor data based on the position information in the bitmask.

### Claim 16 (depends on 13)

The method of claim 13, wherein the generating the sparse mask comprises:
quantizing the low-magnitude tensor data in the tensor by applying the set of quantization parameters to the tensor to obtain a temporary tensor, wherein tensor data in the temporary tensor uses a reduced bit-depth than tensor data in the tensor; and
merging the temporary tensor with the bitmask.

### Claim 17 (depends on 13)

The method of claim 13, wherein a bit-depth of tensor data in the condensed tensor is higher than a bit-depth of tensor data in the bitmask, but lower than a bit-depth of tensor data in the tensor.

### Claim 18 (depends on 13)

The method of claim 13, wherein each entry in the sparse mask comprises two or more bits, with a first bit representing a sign of a corresponding tensor data.

### Claim 19 (depends on 18)

The method of claim 18, wherein the first bit is combined with each of subsequent bits of the two or more bits to implement multi-ternary representation of quantized tensor data.

### Claim 20

A method for sparse decoding a sparse representation of a tensor, comprising:
obtaining, from a shared memory, the sparse representation of the tensor that comprises:
a condensed tensor representing quantized high-magnitude tensor data from the tensor,
a decoding bitmask comprising (1) original position information of the high-magnitude tensor data in the tensor and (2) quantized low-magnitude tensor data from the tensor, and
a set of quantization parameters;
dequantizing, using the set of quantization parameters, the condensed tensor to obtain a dequantized high-magnitude tensor;
dequantizing, using the set of quantization parameters, the decoding bitmask to obtain a dequantized low-magnitude tensor;
generating, using a scatter circuit, a sparse uncondensed tensor based on the dequantized high-magnitude tensor and the decoding bitmask;
merging the sparse uncondensed tensor with the dequantized low-magnitude tensor.
