# US 2021/0342694 A1 · Machine Learning Network Model Compression

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0342694 A1
- Document ID: US20210342694A1
- Status: Published application
- Kind: A1
- Application number: 17/371,590
- Filed: 2021-07-09
- Published: 2021-11-04
- Earliest claimed: 2019-01-09
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Jiafeng Zhu; Wei Wei; Jianle Chen; Wei Wang; Jie Shen
- Web page: https://patentgrep.com/patent/US20210342694A1

## CPC

- G06N 3/063: using electronic means
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- H03M 7/6005: {Decoder aspects}
- G06F 11/34: Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation {; Recording or statistical evaluation of user activity, e.g. usability assessment}
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A first aspect relates to a computer-implemented method for performing model compression. The method includes compressing a machine learning (ML) network model comprising a multiple layer structure to produce a compressed ML network model. The compressed ML network model maintains the multiple layer structure of the ML network model. The method generates a model file for the compressed ML network model. The model file includes the compressed ML network model and decoding information for enabling the ML network model to be decompressed and executed layer-by-layer.

## Claims

### Claim 1

A computer-implemented method for performing model compression, the method comprising:
compressing a machine learning (ML) network model comprising a multiple layer structure to produce a compressed ML network model, the compressed ML network model maintaining the multiple layer structure of the ML network model; and
generating a model file for the compressed ML network model, the model file comprising the compressed ML network model and decoding information for enabling the ML network model to be decompressed and executed layer-by-layer.

### Claim 2 (depends on 1)

The method of claim 1, wherein the decoding information is stored in a general header of the model file.

### Claim 3 (depends on 2)

The method of claim 2, wherein the general header is a first eight bytes of the model file.

### Claim 4 (depends on 2)

The method of claim 2, wherein the general header provides a version, total header length, encoding type, framework type, and serialization type.

### Claim 5 (depends on 1)

The method of claim 1, wherein the model file further comprises a serialization header.

### Claim 6 (depends on 2)

The method of claim 2, wherein the decoding information in the general header comprises a decode layer indication, wherein the decode layer indication is associated with a decode layer resource allocation of layer weights.

### Claim 7 (depends on 6)

The method of claim 6, wherein the decode layer resource allocation is a buffer allocation of a shared buffer for storing layer weights for the multiple layer structure of the ML network model.

### Claim 8 (depends on 6)

The method of claim 6, wherein the decode layer indication indicates that the layer weights of the multiple layer structure share a dynamically allocated buffer in a decoding order of the ML network model.

### Claim 9 (depends on 6)

The method of claim 6, wherein the decode layer indication indicates that the layer weights of the ML network model will each be allocated with respective buffer resources in a buffer.

### Claim 10 (depends on 6)

The method of claim 6, wherein the decode layer indication is carried in a decode layer shared buffer header field of the general header of the model file.

### Claim 11 (depends on 1)

The method of claim 1, further comprising transmitting the model file to a limited resource system.

### Claim 12 (depends on 1)

The method of claim 1, wherein the ML network model is a deep neural network (DNN).

### Claim 13

A computer-implemented method for executing a machine learning (ML) network model, the method comprising:
obtaining a model file comprising a compressed ML network model and decoding information for enabling layer-by-layer decompression and execution, the decoding information comprising a layer-by-layer flag;
determining whether the layer-by-layer flag is set in the model file;
responsive to the layer-by-layer flag being set in the model file, decompressing an Nth layer of the compressed ML network model to generate a decompressed Nth layer of the ML network model, where N is an integer variable representing a layer sequence of the ML network;
loading a set of weighting values for the decompressed Nth layer of the ML network model into a buffer;
installing the set of weighting values from the buffer into the decompressed Nth layer of the ML network model, with each neuron of the decompressed Nth layer of the ML network model receiving a corresponding weighting value; and
processing a set of data through the neurons of the decompressed Nth layer of the ML network model.

### Claim 14 (depends on 13)

The method of claim 13, wherein the decoding information is stored in a general header of the model file.

### Claim 15 (depends on 14)

The method of claim 14, wherein the general header is a first eight bytes of the model file.

### Claim 16 (depends on 14)

The method of claim 14, wherein the general header provides a version, total header length, encoding type, framework type, and serialization type.

### Claim 17 (depends on 13)

The method of claim 13, wherein the model file further comprises a serialization header.

### Claim 18 (depends on 14)

The method of claim 14, wherein the decoding information in the general header comprises a decode layer indication, wherein the decode layer indication is associated with a decode layer resource allocation of layer weights.

### Claim 19 (depends on 18)

The method of claim 18, wherein the decode layer resource allocation is a buffer allocation of a shared buffer for storing layer weights of multiple layers.

### Claim 20 (depends on 19)

The method of claim 19, wherein the decode layer indication indicates that the layer weights of multiple layers share a dynamically allocated buffer in an order of model decoding.

### Claim 21 (depends on 18)

The method of claim 18, wherein the decode layer indication is carried in a decode layer shared buffer field of the general header.

### Claim 22 (depends on 13)

The method of claim 13, further comprising:
discarding the decompressed Nth layer of the ML network model from memory;
determining whether there are additional unprocessed layers of the ML network;
responsive to a determination that there are additional unprocessed layers of the ML network:
incrementing N and generating a N+1 index value;
decompressing the N+1 layer of the compressed ML network model to generate the decompressed N+1 layer of the ML network model;
loading weighting values for the decompressed N+1 layer of the ML network model into the buffer;
installing the weighting values from the buffer into the decompressed N+1 layer of the ML network model; and
processing a second set of data through neurons of the decompressed N+1 layer of the ML network model.

### Claim 23 (depends on 13)

The method of claim 13, wherein the ML network is a deep neural network (DNN).

### Claim 24 (depends on 13)

The method of claim 13, wherein the buffer is a circular buffer or a first-in first-out (FIFO) queue.

### Claim 25 (depends on 13)

The method of claim 13, wherein the layer-by-layer flag is updated when the ML network model is compressed.

### Claim 26 (depends on 13)

The method of claim 13, wherein the layer-by-layer flag is decompressed separately from and before decompression of the Nth layer of the compressed ML network model.

### Claim 27 (depends on 13)

The method of claim 13, wherein the compressed ML network model is not decompressed if the layer-by-layer flag indicates no layer-by-layer operation is supported.

### Claim 28 (depends on 13)

The method of claim 13, further comprising storing the compressed ML network model in a non-volatile memory of a device, and storing the decompressed Nth layer of the ML network model in a volatile memory of the device.

### Claim 29 (depends on 13)

The method of claim 13, further comprising storing both the compressed ML network model and the decompressed Nth layer of the ML network model in a non-volatile memory of a device.
