# US 2023/0418579 A1 · Compilation method and apparatus with neural network

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0418579 A1
- Document ID: US20230418579A1
- Status: Published application
- Kind: A1
- Application number: 18/464,044
- Filed: 2023-09-08
- Published: 2023-12-28
- Earliest claimed: 2021-09-17
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Hanwoong JUNG
- Web page: https://patentgrep.com/patent/US20230418579A1

## CPC

- G06F 8/443: {Optimisation}
- G06F 8/451: {Code distribution (considering CPU load at run-time G06F9/505; load rebalancing G06F9/5083)}
- G06N 3/10: Interfaces, programming languages or software development kits, e.g. for simulating neural networks
- G06N 3/04: Architecture, e.g. interconnection topology
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A compile method for a neural network, the compile method includes receiving data related to the neural network, generating a grouped layer by grouping layers comprised in the neural network based on the data, generating a set of passes executable in parallel based on a dependency between a plurality of passes to process the neural network, generating a set of threads performing a plurality of optimization functions based on whether optimization operations performed by the optimization functions is performed independently for the layers, respectively, or sequentially based on a dependency between the layers, and performing compilation in parallel based on the grouped layer, the set of passes, and the set of threads.

## Claims

### Claim 1

A processor-implemented compile method for a neural network, the compile method comprising:
generating a set of passes executable in parallel based on a dependency between a plurality of passes to process the neural network;
generating a set of threads performing a plurality of optimization functions based on application of optimization operations to layers of the neural network performed by the optimization functions; and
performing compilation in parallel based on a grouped layer of the neural network, the set of passes, and the set of threads.

### Claim 2 (depends on 1)

The compile method of claim 1, further comprising: generating the grouped layer based on the dependency between the layers.

### Claim 3 (depends on 1)

The compile method of claim 1, wherein the generating of the set of threads comprises generating the set of threads based on whether the optimization operations is performed independently for the layers, respectively, or sequentially based on a dependency between the layers.

### Claim 4 (depends on 1)

The compile method of claim 1, wherein the performing of the compiling comprises generating an intermediate representation (IR) by compiling the grouped layer; and
inserting the IR when compiling a second top-level network of the neural network.

### Claim 5 (depends on 1)

The compile method of claim 1, wherein the generating of the set of passes comprises combining passes having no dependency among the passes.

### Claim 6 (depends on 4)

The compile method of claim 4, wherein the generating of the set of threads comprises:
parallelizing neural network operations in response to a portion of the layers having dependency; and
parallelizing each of the layers sequentially in response to the layers having no dependency.

### Claim 7 (depends on 4)

The compile method of claim 4, wherein the generating of the set of threads comprises performing pass fusing on the passes of the layers having no dependency.

### Claim 8 (depends on 4)

The compile method of claim 4, wherein the generating of the set of threads comprises applying multi-threading to an operation of weight compression of the neural network or an operation of weight preprocessing of the neural network.

### Claim 9 (depends on 1)

The compile method of claim 1, further comprising:
in response to a change in a portion of the layers, performing recompiling on a grouped layer comprising the portion of the layers.

### Claim 10 (depends on 1)

A non-transitory computer-readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the compile method of claim 1.

### Claim 11

A compile apparatus for a neural network, the compile apparatus comprising:
a memory; and
a processor configured to
generate a set of passes executable in parallel based on a dependency between a plurality of passes to process the neural network,
generate a set of threads performing a plurality of optimization functions based on application of optimization operations to layers of the neural network performed by the optimization functions, and
perform compilation in parallel based on a grouped layer of the neural network, the set of passes, and the set of threads.

### Claim 12 (depends on 11)

The compile apparatus of claim 11, wherein the processor is further configured to generate the grouped layer based on the dependency between the layers.

### Claim 13 (depends on 11)

The compile apparatus of claim 11, wherein the processor is further configured to the set of threads based on whether the optimization operations is performed independently for the layers, respectively, or sequentially based on a dependency between the layers.

### Claim 14 (depends on 11)

The compile apparatus of claim 11, wherein the processor is further configured to
generate an intermediate representation (IR) by compiling the grouped layer, and
perform compilation by inserting the IR when compiling a second top-level network of the neural network.

### Claim 15 (depends on 11)

The compile apparatus of claim 11, wherein the processor is further configured to generate the set of passes by combining passes having no dependency among the passes.

### Claim 16 (depends on 13)

The compile apparatus of claim 13, wherein the processor is further configured to
generate the set of threads by parallelizing neural network operations in response to a portion of the layers having a dependency, and
generate the set of threads by parallelizing each of the layers sequentially in response to the layers having no dependency.

### Claim 17 (depends on 13)

The compile apparatus of claim 13, wherein the processor is further configured to perform pass fusing on the passes of the layers having no dependency.

### Claim 18 (depends on 13)

The compile apparatus of claim 13, wherein the processor is further configured to generate the set of threads by applying multi-threading to an operation of weight compression of the neural network or an operation of weight preprocessing of the neural network.

### Claim 19 (depends on 11)

The compile apparatus of claim 11, wherein in response to a change in a portion of the layers, the processor is further configured to perform recompiling on a grouped layer comprising the portion of the layers.
