# US 2021/0081773 A1 · Variable clock adaptation in neural network processors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0081773 A1
- Document ID: US20210081773A1
- Status: Published application
- Kind: A1
- Application number: 17/023,144
- Filed: 2020-09-16
- Published: 2021-03-18
- Earliest claimed: 2019-09-18
- Assignees: STMicroelectronics S.r.l.; STMicroelectronics International N.V.
- Applicants: STMICROELECTRONICS S.r.l.; STMicroelectronics International N.V.
- Inventors: Nitin CHAWLA; Giuseppe DESOLI; Manuj AYODHYAWASI; Thomas BOESCH; Surinder Pal SINGH
- Web page: https://patentgrep.com/patent/US20210081773A1

## CPC

- G06N 3/063: using electronic means
- G06F 1/08: Clock generators with changeable or programmable clock frequency
- G06F 1/3228: Monitoring task completion, e.g. by use of idle timers, stop commands or wait commands
- G06F 1/324: by lowering clock frequency
- G06F 1/3296: by lowering the supply or operating voltage
- G06F 9/5027: {the resource being a machine, e.g. CPUs, Servers, Terminals}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE

## Abstract

Systems and devices are provided to increase computational and/or power efficiency for one or more neural networks via a computationally driven closed-loop dynamic clock control. A clock frequency control word is generated based on information indicative of a current frame execution rate of a processing task of the neural network and a reference clock signal. A clock generator generates the clock signal of neural network based on the clock frequency control word. A reference frequency may be used to generate the clock frequency control word, and the reference frequency may be based on information indicative of a sparsity of data of a training frame.

## Claims

### Claim 1

A device, comprising:
control word generation circuitry, which, in operation, generates a clock frequency control word based on information indicative of a current frame execution rate of a processing task of a neural network and a reference clock signal; and
a clock generator coupled to the control word generation circuitry and which, in operation, generates a neural network clock signal for use by the neural network based on the clock frequency control word.

### Claim 2 (depends on 1)

The device of claim 1, wherein the control word generation circuitry, in operation, determines the frequency control word based on a reference frequency.

### Claim 3 (depends on 2)

The device of claim 2, wherein the information indicative of the current frame execution rate comprises frame synchronization information and the control word generation circuitry comprises:
a frequency counter which, in operation, determines a current frame processing frequency based at least in part on the reference clock signal and the frame synchronization information; and
a proportional integral controller which, in operation, generates the clock frequency control word based on a difference between the determined frame processing frequency and the reference frequency.

### Claim 4 (depends on 3)

The device of claim 3, wherein the reference frequency is based on a frame processing frequency of an execution of a kernel processing only non-zero operations.

### Claim 5 (depends on 4)

The device of claim 4, wherein the reference frequency corresponds to a nominal kernel data sparsity associated with the processing task of the neural network.

### Claim 6 (depends on 2)

The device of claim 2, wherein the reference frequency is based on an average frame execution rate.

### Claim 7 (depends on 3)

The device of claim 3, wherein the control word generating circuitry comprises a frequency error generator coupled to the frequency counter, which, in operation, determines the difference between the determined frame processing frequency and the reference frequency.

### Claim 8 (depends on 1)

The device of claim 1, comprising an adaptive voltage scaling controller which, in operation, modifies a supply voltage of the neural network.

### Claim 9 (depends on 8)

The device of claim 8, wherein, in operation, the adaptive voltage scaling controller modifies the supply voltage based on the neural network clock signal.

### Claim 10 (depends on 1)

The device of claim 1, comprising a dynamic voltage frequency scaling controller which, in operation, generates control information, and the clock generator generates the neural network clock signal based on the control information generated by the dynamic voltage frequency scaling controller.

### Claim 11 (depends on 1)

The device of claim 1, wherein the information indicative of a current frame execution rate includes a plurality of start interrupt times associated with the processing task and a corresponding plurality of stop interrupt times associated with the processing task.

### Claim 12 (depends on 11)

The device of claim 11, wherein each of the plurality of start interrupt times corresponds to a starting time associated with processing a respective frame of data and each of the plurality of stop interrupt times corresponds to a stopping time associated with processing the respective frame of data.

### Claim 13 (depends on 12)

The device of claim 12, wherein each of the plurality of start interrupt times and stop interrupt times is associated with a processing weight, and the control word generation circuitry, in operation, generates the clock frequency control word based on the processing weights.

### Claim 14 (depends on 1)

The device of claim 1, wherein the information indicative of the current frame execution rate is associated with a current kernel data sparsity of the processing task of the neural network.

### Claim 15 (depends on 1)

The device of claim 1, wherein the clock generator is one of a group that includes a phase lock loop (PLL) clock generator and a frequency lock loop (FLL) clock generator.

### Claim 16

A system, comprising:
neural network accelerator circuitry; and
closed-loop clock-rate control circuitry coupled to the neural network accelerator circuitry, and which, in operation, generates a clock signal of the neural network accelerator circuitry, the closed-loop clock-rate control circuitry including:
control-word generation circuitry, which, in operation, generates a clock frequency control word based on information indicative of a current frame execution rate of a processing task of the neural network accelerator circuitry and a reference clock signal; and
a clock generator coupled to the control word generation circuitry and which, in operation, generates the clock signal of neural network accelerator circuitry based on the clock frequency control word.

### Claim 17 (depends on 16)

The system of claim 16, wherein the control-word generation circuitry, in operation, generates the clock frequency control word based on a reference frequency.

### Claim 18 (depends on 16)

The system of claim 16, comprising:
an adaptive voltage scaling controller, which, in operation, modifies a supply voltage of the neural network.

### Claim 19 (depends on 16)

The system of claim 16, comprising:
a dynamic voltage frequency scaling controller, which, in operation, generates dynamic clock control information, wherein the clock generator generates the clock signal of the neural network accelerator circuitry based on the dynamic clock control information.

### Claim 20

A method, comprising:
executing a kernel using a neural network accelerator; and
performing closed-loop clock-rate control of an operating frequency of the neural network accelerator during execution of the kernel, the performing closed-loop clock-rate control including:
generating a clock frequency control word based on information indicative of a current frame execution rate of the kernel and a reference clock signal; and
generating a clock signal of neural network accelerator circuitry based on the clock frequency control word.

### Claim 21 (depends on 20)

The method of claim 20, wherein the clock frequency control word is generated based on a reference frequency.

### Claim 22 (depends on 20)

The method of claim 20, comprising:
modifying a supply voltage of the neural network based on the generated clock signal of the neural network accelerator circuitry.

### Claim 23 (depends on 20)

The method of claim 20, comprising:
modifying the operating frequency of the neural network using a dynamic voltage frequency scaling controller.
