# US 2020/0401882 A1 · Learning neural networks of programmable device blocks directly with backpropagation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0401882 A1
- Document ID: US20200401882A1
- Status: Published application
- Kind: A1
- Application number: 16/449,264
- Filed: 2019-06-21
- Published: 2020-12-24
- Earliest claimed: 2019-06-21
- Assignees: Xilinx, Inc.
- Applicants: Xilinx, Inc.
- Inventors: Yaman Umuroglu; Nicholas Fraser; Michaela Blott; Kristof Denolf; Kornelis A. Vissers
- Web page: https://patentgrep.com/patent/US20200401882A1

## CPC

- G06N 3/08: Learning methods
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/0499: Feedforward networks
- G06N 3/063: using electronic means
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 3/09: Supervised learning
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

An example method of training a neural network includes defining hardware building blocks (HBBs), neuron equivalents (NEQs), and conversion procedures from NEQs to HBBs; defining the neural network using the NEQs in a machine learning framework; training the neural network on a training platform; and converting the neural network as trained into a netlist of HBBs using the conversion procedures to convert the NEQs in the neural network to the HBBs of the netlist.

## Claims

### Claim 1

A method of training a neural network, comprising:
defining hardware building blocks (HBBs), neuron equivalents (NEQs), and conversion procedures from NEQs to HBBs;
defining the neural network using the NEQs in a machine learning framework;
training the neural network on a training platform; and
converting the neural network as trained into a netlist of HBBs using the conversion procedures to convert the NEQs in the neural network to the HBBs of the netlist.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
post-processing the netlist to perform one or more optimizations of the netlist.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
generating an implementation of the netlist for programming a programmable device.

### Claim 4 (depends on 1)

The method of claim 1, wherein the step of defining the HBBs, the NEQs, and the conversion procedures comprises:
identifying primitives in a programmable fabric of a programmable device to implement the HBBs;
for each HBB, defining one or more of the NEQs in the machine learning framework; and
for each NEQ, identifying a conversion procedure to a corresponding HBB.

### Claim 5 (depends on 1)

The method of claim 1, wherein each NEQ comprises a dot product operation and a nonlinear operation.

### Claim 6 (depends on 5)

The method of claim 5, wherein each NEQ includes: quantized inputs and quantized outputs, where a sum of widths of the quantized inputs is less than or equal to a number of address bits of a memory component implementing an HBB to which the respective NEQ converts, and where a sum of widths of the quantized outputs is less than or equal to a width of the memory component.

### Claim 7 (depends on 5)

The method of claim 5, wherein each NEQ further comprises one or more operations between the dot product operation and the nonlinear operation.

### Claim 8 (depends on 1)

The method of claim 1, wherein each HBB comprises a memory component in a programmable fabric of a programmable device.

### Claim 9

A non-transitory computer readable medium having stored thereon instructions that when executed by a processor cause the processor to perform a method of training a neural network, comprising:
defining hardware building blocks (HBBs), neuron equivalents (NEQs), and conversion procedures from NEQs to HBBs;
defining the neural network using the NEQs in a machine learning framework;
training the neural network on a training platform; and
converting the neural network as trained into a netlist of HBBs using the conversion procedures to convert the NEQs in the neural network to the HBBs of the netlist.

### Claim 10 (depends on 9)

The non-transitory computer readable medium of claim 9, further comprising:
post-processing the netlist to perform one or more optimizations of the netlist.

### Claim 11 (depends on 9)

The non-transitory computer readable medium of claim 9, further comprising:
generating an implementation of the netlist for programming a programmable device.

### Claim 12 (depends on 9)

The method of claim 9, wherein the step of defining the HBBs, the NEQs, and the conversion procedures comprises:
identifying primitives in a programmable fabric of a programmable device to implement the HBBs;
for each HBB, defining one or more of the NEQs in the machine learning framework; and
for each NEQ, identifying a conversion procedure to a corresponding HBB.

### Claim 13 (depends on 9)

The non-transitory computer readable medium of claim 9, wherein each NEQ comprises a dot product operation and a nonlinear operation, and wherein each NEQ includes: quantized inputs and quantized outputs, where a sum of widths of the quantized inputs is less than or equal to a number of address bits of a memory component implementing an HBB to which the respective NEQ converts, and where a sum of widths of the quantized outputs is less than or equal to a width of the memory component.

### Claim 14 (depends on 13)

The non-transitory computer readable medium of claim 13, wherein each NEQ further comprises one or more operations between the dot product operation and the nonlinear operation.

### Claim 15 (depends on 9)

The non-transitory computer readable medium of claim 9, wherein each HBB comprises a memory component in a programmable fabric of a programmable device.

### Claim 16

A computing system, comprising:
a system memory configured to store code implementing design tools;
a training platform; and
a processor coupled to the system memory and the training platform, the processor configured to execute the code to perform training of a neural network by:
defining hardware building blocks (HBBs), neuron equivalents (NEQs), and conversion procedures from NEQs to HBBs;
defining the neural network using the NEQs in a machine learning framework;
training the neural network on the training platform; and
converting the neural network as trained into a netlist of HBBs using the conversion procedures to convert the NEQs in the neural network to the HBBs of the netlist.

### Claim 17 (depends on 16)

The computing system of claim 16, wherein the training of the neural network further comprises:
post-processing the netlist to perform one or more optimizations of the netlist.

### Claim 18 (depends on 16)

The computing system of claim 16, wherein the training of the neural network further comprises:
generating an implementation of the netlist for programming a programmable device.

### Claim 19 (depends on 16)

The computing system of claim 16, wherein the defining the HBBs, the NEQs, and the conversion procedures comprises:
identifying primitives in a programmable fabric of a programmable device to implement the HBBs;
for each HBB, defining one or more of the NEQs in the machine learning framework; and
for each NEQ, identifying a conversion procedure to a corresponding HBB.

### Claim 20 (depends on 16)

The computing system of claim 16, wherein each NEQ comprises a dot product operation and a nonlinear operation, and wherein each NEQ includes: quantized inputs and quantized outputs, where a sum of widths of the quantized inputs is less than or equal to a number of address bits of a memory component implementing an HBB to which the respective NEQ converts, and where a sum of widths of the quantized outputs is less than or equal to a width of the memory component.

### Claim 21 (depends on 16)

The computing system of claim 16, wherein each HBB comprises a memory component in a programmable fabric of a programmable device.