# US 2024/0005159 A1 · Simplification device and simplification method for neural network model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005159 A1
- Document ID: US20240005159A1
- Status: Published application
- Kind: A1
- Application number: 17/892,145
- Filed: 2022-08-22
- Published: 2024-01-04
- Earliest claimed: 2022-06-30
- Assignees: Neuchips Corporation
- Applicants: Neuchips Corporation
- Inventors: Po-Han Chen; Yi Lee; Kai-Chiang Wu; Youn-Long Lin; Juinn-Dar Huang
- Web page: https://patentgrep.com/patent/US20240005159A1

## CPC

- G06N 3/045: Combinations of networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A simplification device and a simplification method for neural network model are provided. The simplification method may simplify an original trained neural network model to a simplified trained neural network model, wherein the simplified trained neural network model includes at most two linear operation layers. The simplification method includes: converting the original trained neural network model into an original mathematical function; performing an iterative analysis operation on the original mathematical function to simplify the original mathematical function to a simplified mathematical function, wherein the simplified mathematical function has a new weight; computing the new weight by using multiple original weights of the original trained neural network model; and converting the simplified mathematical function to the simplified trained neural network model.

## Claims

### Claim 1

A simplification method for neural network model, configured to simplify an original trained neural network model to a simplified trained neural network model, wherein the simplified trained neural network model comprises at most two linear operation layers, and the simplification method for neural network model comprises:
receiving the original trained neural network model;
calculating a first new weight of the at most two linear operation layers of the simplified trained neural network model by using a plurality of original weights of the original trained neural network model; and
generating the simplified trained neural network model based on the first new weight.

### Claim 2 (depends on 1)

The simplification method for neural network model as claimed in claim 1, wherein the simplified trained neural network model is denoted as y=x@WI+BI, y represents an output of the simplified trained neural network model, @ represents any linear operation of the simplified trained neural network model, x represents an input of the simplified trained neural network model, WI represents the first new weight, and BI represents a new bias of the simplified trained neural network model.

### Claim 3 (depends on 2)

The simplification method for neural network model as claimed in claim 2, wherein the any linear operation @ comprises a matrix multiply-accumulate operation.

### Claim 4 (depends on 2)

The simplification method for neural network model as claimed in claim 2, wherein the original trained neural network model is denoted as y=(x@w1+b1)@w2+b2, w1 and b1 respectively represent an original weight and an original bias of a first linear operation layer of the original trained neural network model, w2 and b2 respectively represent an original weight and an original bias of a second linear operation layer of the original trained neural network model, and the simplification method further comprises:
calculating WI=w1@w2 to determine the first new weight WI of the simplified trained neural network model; and
calculating BI=b1@w2+b2 to determine the new bias BI of the simplified trained neural network model.

### Claim 5 (depends on 1)

The simplification method for neural network model as claimed in claim 1, further comprising:
calculating a second new weight of the at most two linear operation layers of the simplified trained neural network model by using at least one original weight of the original trained neural network model, wherein the simplified trained neural network model is denoted as y=WII@(x@WI+BI), y represents an output of the simplified trained neural network model, @ represents any linear operation of the simplified trained neural network model, WII represents the second new weight, x represents an input of the simplified trained neural network model, WI represents the first new weight, and BI represents a new bias of the simplified trained neural network model; and
calculating the second new weight BI of the simplified trained neural network model by using at least one original weight and at least one original bias of the original trained neural network model.

### Claim 6 (depends on 5)

The simplification method for neural network model as claimed in claim 5, wherein the original trained neural network model is denoted as y=((x@w1+b1)T@w2+b2)T@w3, ( )T represents a matrix transpose operation, w1 and b1 respectively represent an original weight and an original bias of a first linear operation layer of the original trained neural network model, w2 and b2 respectively represent an original weight and an original bias of a second linear operation layer of the original trained neural network model, w3 represents an original weight of a third linear operation layer of the original trained neural network model, and the simplification method further comprises:
calculating WII=(w2)T to determine the second new weight WII of the simplified trained neural network model;
calculating WI=w1@w3 to determine the first new weight WI of the simplified trained neural network model; and
calculating BI=b1@w3+((w2)T)−1@(b2)T@w3 to determine the bias BI of the simplified trained neural network model.

### Claim 7 (depends on 1)

The simplification method for neural network model as claimed in claim 1, further comprising:
receiving the original trained neural network model;
converting the original trained neural network model into an original mathematical function;
performing an iterative analysis operation on the original mathematical function to simplify the original mathematical function to a simplified mathematical function, wherein the simplified mathematical function has the first new weight; and
converting the simplified mathematical function to the simplified trained neural network model.

### Claim 8 (depends on 7)

The simplification method for neural network model as claimed in claim 7, wherein the original mathematical function is denoted as y=(( . . . ((xT0@w1+b1)T1@w2+b2)T2 . . . )Tn-1@wn+bn)Tn, y represents an output of the original mathematical function, x represents an input of the original mathematical function, T0 represents whether to transpose the input x, @ represents any linear operation of neural network model, w1 and b1 respectively represent an original weight and an original bias of a first linear operation layer of the original trained neural network model, T1 represents whether to transpose a result of the first linear operation layer, w2 and b2 respectively represent an original weight and an original bias of a second linear operation layer of the original trained neural network model, T2 represents whether to transpose a result of the second linear operation layer, Tn−1 represents whether to transpose a result of an (n−1)th linear operation layer of the original trained neural network model, wn and bn respectively represent an original weight and an original bias of an nth linear operation layer of the original trained neural network model, Tn represents whether to transpose a result of the nth linear operation layer, and n is an integer greater than 1.

### Claim 9 (depends on 8)

The simplification method for neural network model as claimed in claim 8, wherein the iterative analysis operation comprises n iterations, and a first iteration of the n iterations comprises:
taking the input x of the original mathematical function as a starting point, extracting (xT0@w1+b1)T1 corresponding to the first linear operation layer from the original mathematical function;
defining X1 as x;
checking T0;
defining F1 as transposed X1 when T0 represents “transpose”, defining F′1 as F1@w1+b1, and checking T1;
defining Y1 as transposed F′1 when T0 represents “transpose” and T1 represents “transpose”, so that Y1=(w1)T@X1+(b1)T, where ( )T represents a transpose operation;
defining Y1 as F′1 when T0 represents “transpose” and T1 represents “not transpose”, so that Y1=(X1)T@w1+b1;
defining F1 as X1 when T0 represents “not transpose”, defining F′1 as F1@w1+b1, and checking T1;
defining Y1 as transposed F′1 when T0 represents “not transpose” and T1 represents “transpose”, so that Y1=(w1)T@(X1)T+(b1)T;
defining Y1 as F′1 when T0 represents “not transpose” and T1 represents “not transpose” such that Y1=X1@w1+b1; and
replacing (xT0@w1+b1)T1 in the original mathematical function with Y1.

### Claim 10 (depends on 9)

The simplification method for neural network model as claimed in claim 9, wherein a second iteration of the n iterations comprises:
extracting (Y1@w2+b2)T2 corresponding to the second linear operation layer from the original mathematical function;
defining X2 as Y1;
defining F2 as X2;
defining F′2 as F2@w2+b2;
checking T2;
defining Y2 as transposed F′2 when T2 represents “transpose”, so that Y2=(w2)T@(X2)T+(b2)T;
defining Y2 as F′2 when T2 represents “not transpose”, such that Y2=X2@W2+b2; and
replacing (Y1@w2+b2)T2 in the original mathematical function with Y2.

### Claim 11 (depends on 8)

The simplification method for neural network model as claimed in claim 8, wherein the iterative analysis operation comprises n iterations, the simplified mathematical function is generated after the n iterations are completed, and the simplified mathematical function is denoted as y=WII@(x@WI+BI)+BII, where WI represents the first new weight, and the iterative analysis operation uses some or all of the original weights w1 to wn to pre-calculate a first constant to serve as the first new weight WI; WII represents a second new weight of the at most two linear operation layers, and the iterative analysis operation uses at least one of the original weights w1 to wn to pre-calculate a second constant to serve as the second new weight WII; BI represents a first new bias of the at most two linear operation layers, and the iterative analysis operation uses at least one of the original weights w1 to wn and at least one of the original biases b1 to bn to pre-calculate a third constant to serve as the first new bias BI; BII represents a second new bias of the at most two linear operation layers, and the iterative analysis operation uses “at least one of the original weights w1 to wn” or “at least one of the original biases b1 to bn” or “at least one of the original weights w1 to wn and at least one of the original biases b1 to bn” to pre-calculate a fourth constant to serve as the second new bias BII.

### Claim 12 (depends on 1)

A simplification device for neural network model, comprising:
a memory, storing a computer readable program; and
a processor, coupled to the memory to execute the computer readable program;
wherein the processor executes the computer readable program to realize the simplification method for neural network model as claimed in claim 1.

### Claim 13 (depends on 1)

A non-transitory storage medium, for storing a computer readable program, wherein the computer readable program is executed by a computer to realize the simplification method for neural network model as claimed in claim 1.
