# US 2022/0012164 A1 · Systems, apparatus, and methods to debug accelerator hardware

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0012164 A1
- Document ID: US20220012164A1
- Status: Published application
- Kind: A1
- Application number: 17/483,521
- Filed: 2021-09-23
- Published: 2022-01-13
- Earliest claimed: 2021-09-23
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Martin-Thomas Grymel; David Bernard; Martin Power; Niall Hanrahan; Kevin Brady
- Web page: https://patentgrep.com/patent/US20220012164A1

## CPC

- G06F 11/3648: {using additional hardware}
- G06F 11/3652: {in-circuit-emulation [ICE] arrangements}
- G06F 11/277: with comparison between actual response and known fault-free response
- G06F 11/3075: {the data filtering being achieved in order to maintain consistency among the monitored data, e.g. ensuring that the monitored data belong to the same timeframe, to the same system or component}
- G06F 11/3656: {using a specific debug interface}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/063: using electronic means
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/08: Learning methods
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed to debug a hardware accelerator such as a neural network accelerator for executing Artificial Intelligence computational workloads. An example apparatus includes a core with a core input and a core output to execute executable code based on a machine-learning model to generate a data output based on a data input, and debug circuitry coupled to the core. The debug circuitry is configured to detect a breakpoint associated with the machine-learning model, compile executable code based on at least one of the machine-learning model or the breakpoint. In response to the triggering of the breakpoint, the debug circuitry is to stop the execution of the executable code and output data such as the data input, data output and the breakpoint for debugging the hardware accelerator.

## Claims

### Claim 1

An apparatus to debug a hardware accelerator, the apparatus comprising:
a core with a core input and a core output, the core to execute executable code to generate a data output based on a data input, the executable code based on a machine-learning model; and
debug circuitry coupled to at least one of the core input or the core output, the debug circuitry including:
interface circuitry with at least one of an interface input coupled to the core output or an interface output coupled to the core input, the interface circuitry to receive at least one of the data input or the data output;
multiplexer circuitry with a multiplexer input and a multiplexer output, the multiplexer input coupled to the interface output; and
a shift register with a shift register input coupled to the multiplexer output, the shift register to output the at least one of the data input or the data output in response to a trigger of a breakpoint associated with an execution of the executable code.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the interface input is coupled to memory, the interface output is coupled to the core input, and wherein:
the interface circuitry is to:
receive the data input from the memory; and
provide the data input to the core input in response to the breakpoint not being triggered; and
the core is to provide the data output from the core output to the memory.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the core input is coupled to memory, the interface input is coupled to the core output, the interface output is coupled to the memory, and wherein:
the core is to:
receive the data input from the memory; and
provide the data output from the core output to the interface input; and
the interface circuitry is to provide the data output to the memory in response to the breakpoint not being triggered.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the debug circuitry is first debug circuitry, the interface circuitry is first interface circuitry, the interface input is a first interface input, the interface output is a first interface output, the first interface input is coupled to memory, the first interface circuitry to receive the data input from the memory, and further including:
the first interface circuitry is to provide the data input to the core in response to the breakpoint not being triggered;
second debug circuitry including second interface circuitry with a second interface input and a second interface output, the second interface input coupled to the core output, the second interface output coupled to the memory, the second debug circuitry to at least one of:
receive the data output from the core;
output the at least one of the data input or the data output in response to the trigger of the breakpoint; or
output the data output to the memory in response to the breakpoint not being triggered.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the debug circuitry is included in the core, the hardware accelerator is a neural network accelerator, the machine-learning model is a neural network, and wherein:
the core is to execute executable code to generate a data output based on a data input, the executable code including a breakpoint, the executable code based on at least one of the neural network or the breakpoint; and
the debug circuitry is to:
trigger the breakpoint to stop the execution of the executable code; and
output at least one of the data input, the data output, or the breakpoint.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the at least one of the data input or the data output includes a first value, and the debug circuitry includes:
control circuitry to obtain a second value corresponding to the breakpoint from a configuration register of the core;
a breakpoint register coupled to the control circuitry, the breakpoint register to store the second value;
comparator circuitry with a first comparator input and a second comparator input, the first comparator input coupled to the interface output, the second comparator input coupled to the breakpoint register and the control circuitry, the comparator circuitry to compare the first value and the second value; and
the control circuitry to instruct the interface circuitry to provide the at least one of the data input or the data output to the multiplexer circuitry in response to a match of the first value and the second value based on the comparison, the triggering of the breakpoint in response to the match, the control circuitry to instruct the interface circuitry to receive an indication of the match from the comparator circuitry.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the interface circuitry is first interface circuitry, the interface input is a first interface input, the interface output is a first interface output, the core includes a first thread and a second thread, the first thread coupled to the first interface input, and further including:
second interface circuitry with a second interface input and a second interface output, the second interface input coupled to the second thread; and
counter circuitry with a counter output coupled to a selection input of the multiplexer circuitry, the counter circuitry to:
output a first value to instruct the multiplexer circuitry to select the output of the first interface circuitry; and
output a second value to instruct the multiplexer circuitry to select the output of the second interface circuitry.

### Claim 8

An apparatus to debug a hardware accelerator, the apparatus comprising:
means for executing executable code to generate a data output based on a data input, the executable code based on a machine-learning model; and
means for debugging the hardware accelerator, the means for debugging coupled to the means for executing, the means for debugging including:
means for receiving at least one of the data input or the data output, the means for receiving with at least one of an input coupled to an output of the means for executing or an output coupled to an input of the means for executing;
means for selecting the means for receiving, the means for selecting with an input coupled to an output of the means for receiving; and
means for outputting the at least one of the data input or the data output in response to a trigger of a breakpoint associated with an execution of the executable code, the means for outputting with an input coupled to an output of the means for selecting.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the input of the means for receiving is coupled to means for storing, the output of the means for receiving is coupled to the input of the means for executing, and wherein:
the means for receiving is to:
receive the data input from the means for storing; and
provide the data input to the input of the means for executing in response to the breakpoint not being triggered; and
the means for executing is to provide the data output from the output of the means for executing to the means for storing.

### Claim 10 (depends on 8)

The apparatus of claim 8, wherein the input of the means for executing is coupled to means for storing, the input of the means for receiving is coupled to the output of the means for executing, the output of the means for receiving is coupled to the means for storing, and wherein:
the means for executing is to:
receive the data input from the means for storing; and
provide the data output from the output of the means for executing to the input of the means for receiving; and
the means for receiving is to provide the data output to the means for storing in response to the breakpoint not being triggered.

### Claim 11 (depends on 8)

The apparatus of claim 8, wherein the means for debugging is first means for debugging, the means for receiving is first means for receiving, the input of the first means for receiving is coupled to means for storing, the first means for receiving is to receive the data input from the means for storing, and further including:
the first means for receiving is to provide the data input to the means for executing in response to the breakpoint not being triggered;
second means for debugging the hardware accelerator, the second means for debugging including second means for receiving, an input of the second means for receiving coupled to the output of the means for executing, an output of the second means for receiving coupled to the means for storing, the second means for debugging to at least one of:
receive the data output from the means for executing;
output the at least one of the data input or the data output in response to the trigger of the breakpoint; or
output the data output to the means for storing in response to the breakpoint not being triggered.

### Claim 12 (depends on 8)

The apparatus of claim 8, wherein the means for debugging is included in the means for executing, the hardware accelerator is a neural network accelerator, the machine-learning model is a neural network, and wherein:
the means for executing is to execute executable code to generate a data output based on a data input, the executable code including a breakpoint, the executable code based on at least one of the neural network or the breakpoint; and
the means for debugging is to:
trigger the breakpoint to stop the execution of the executable code; and
output at least one of the data input, the data output, or the breakpoint.

### Claim 13 (depends on 8)

The apparatus of claim 8, wherein the at least one of the data input or the data output includes a first value, and the means for debugging includes:
means for controlling the means for debugging, the means for controlling to obtain a second value corresponding to the breakpoint from a first means for storing;
second means for storing the second value, the second means for storing coupled to the means for controlling;
means for comparing to compare the first value and the second value, a first input of the means for comparing coupled to the output of the means for receiving, a second input of the means for comparing coupled to the second means for storing and the means for controlling; and
the means for controlling to control the means for receiving to provide the at least one of the data input or the data output to the means for selecting in response to a match of the first value and the second value based on the comparison, the triggering of the breakpoint in response to the match, the means for controlling to control the means for receiving to receive an indication of the match from the means for comparing.

### Claim 14 (depends on 8)

The apparatus of claim 8, wherein the data input is a first data input, the data output is a first data output, the means for receiving is first means for receiving, the means for executing is a first means for executing, and further including:
second means for receiving at least one of a second data input or a second data output, an input of the second means for receiving coupled to second means for executing; and
means for incrementing a counter, an output of the means for incrementing coupled to a selection input of the means for selecting, the means for incrementing to:
output a first value of the counter to instruct the means for selecting to select the output of the first means for receiving; and
output a second value of the counter to instruct the means for selecting to select an output of the second means for receiving.

### Claim 15

An apparatus to debug a hardware accelerator, the apparatus comprising:
at least one memory;
instructions in the apparatus; and
processor circuitry to at least one of execute or instantiate the instructions to:
generate a breakpoint associated with a machine-learning model;
compile executable code based on at least one of the machine-learning model or the breakpoint, the executable code to be executed by the processor circuitry to generate a data output based on a data input;
in response to an execution of the executable code:
trigger the breakpoint to stop the execution of the executable code; and
output, with debug circuitry included in the processor circuitry, at least one of the data input, the data output, or the breakpoint.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the processor circuitry is to:
identify the breakpoint to be triggered on a per-workload basis;
insert the breakpoint into the executable code to be invoked on the per-workload basis;
in response to the execution of the executable code by a first core of the processor circuitry, halt execution of the executable code by the first core when the breakpoint is triggered by the first core; and
in response to the execution of the executable code by a second core of the processor circuitry, halt execution of the executable code by the second core when the breakpoint is triggered by the second core.

### Claim 17 (depends on 15)

The apparatus of claim 15, wherein the processor circuitry is to:
identify the breakpoint to be triggered on a per-core basis;
identify the breakpoint to be written into a first configuration register of a first core of the processor circuitry and not a second configuration register of a second core of the processor circuitry; and
write the breakpoint into the first configuration register, the triggering of the breakpoint to halt execution of the executable code by the first core while the second core is to continue execution of the executable code.

### Claim 18 (depends on 15)

The apparatus of claim 15, wherein the data input includes first data, the data output includes second data, and the processor circuitry is to:
identify the breakpoint to be triggered based on third data;
write the third data into a configuration register of a core of the processor circuitry;
execute a first comparison of the first data and the third data, the triggering of the breakpoint in response to a first match of the first data and the third data based on the first comparison; and
execute a second comparison of the second data and the third data, the triggering of the breakpoint in response to a second match of the second data and the third data based on the second comparison.

### Claim 19 (depends on 15)

The apparatus of claim 15, wherein the processor circuitry is to:
identify the breakpoint to be triggered based on a first address in memory associated with the data output;
write the first address into a configuration register of a core of the processor circuitry;
in response to executing the executable code, identify a second address in the memory at which to write the data output; and
execute a comparison of the first address and the second address, the triggering of the breakpoint in response to a match of the first address and the second address.

### Claim 20 (depends on 15)

The apparatus of claim 15, wherein the data input is a first data input, and the processor circuitry is to:
in response to the triggering of the breakpoint, obtain a control signal indicative of executing an incremental operation of the executable code, the incremental operation including at least one of a read operation to read a first value, a write operation to write a second value, or a computation operation to determine a third value based on a second data input; and
output at least one of the first value, the second value, or the third value.

### Claim 21 (depends on 15)

The apparatus of claim 15, wherein the processor circuitry is to:
in response to the triggering of the breakpoint, at least one of:
adjust a first value of the data input;
adjust a second value of a first register of a core of the processor circuitry; or
adjust a third value of a second register of the debug circuitry; and
resume the execution of the executable code based on at least one of the first value, the second value, or the third value.

### Claim 22

At least one non-transitory computer readable medium comprising instructions that, when executed, cause first processor circuitry to at least:
generate a breakpoint associated with a machine-learning model;
compile executable code based on at least one of the machine-learning model or the breakpoint, the executable code to be executed by the first processor circuitry or second processor circuitry to generate a data output based on a data input;
in response to an execution of the executable code:
trigger the breakpoint to stop the execution of the executable code; and
output, with debug circuitry included in the first processor circuitry or the second processor circuitry, at least one of the data input, the data output, or the breakpoint.

### Claim 23 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the instructions, when executed, cause the first processor circuitry to:
identify the breakpoint to be triggered on a per-workload basis;
insert the breakpoint into the executable code to be invoked on the per-workload basis;
in response to the execution of the executable code by a first core of the first processor circuitry or the second processor circuitry, halt execution of the executable code by the first core when the breakpoint is triggered by the first core; and
in response to the execution of the executable code by a second core of the first processor circuitry or the second processor circuitry, halt execution of the executable code by the second core when the breakpoint is triggered by the second core.

### Claim 24 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the instructions, when executed, cause the first processor circuitry to:
identify the breakpoint to be triggered on a per-core basis;
identify the breakpoint to be written into a first configuration register of a first core of the first processor circuitry or the second processor circuitry and not a second configuration register of a second core of the first processor circuitry or the second processor circuitry; and
write the breakpoint into the first configuration register, the triggering of the breakpoint to halt execution of the executable code by the first core while the second core is to continue execution of the executable code.

### Claim 25 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the data input includes first data, the data output includes second data, and the instructions, when executed, cause the first processor circuitry to:
identify the breakpoint to be triggered based on third data;
write the third data into a configuration register of a core of the first processor circuitry or the second processor circuitry;
execute a first comparison of the first data and the third data, the triggering of the breakpoint in response to a first match of the first data and the third data based on the first comparison; and
execute a second comparison of the second data and the third data, the triggering of the breakpoint in response to a second match of the second data and the third data based on the second comparison.

### Claim 26 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the instructions, when executed, cause the first processor circuitry to:
identify the breakpoint to be triggered based on a first address in memory associated with the data output;
write the first address into a configuration register of a core of the first processor circuitry or the second processor circuitry;
in response to executing the executable code, identify a second address in the memory at which to write the data output; and
execute a comparison of the first address and the second address, the triggering of the breakpoint in response to a match of the first address and the second address.

### Claim 27 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the data input is a first data input, and the instructions, when executed, cause the first processor circuitry to:
in response to the triggering of the breakpoint, obtain a control signal indicative of executing an incremental operation of the executable code, the incremental operation including at least one of a read operation to read a first value, a write operation to write a second value, or a computation operation to determine a third value based on a second data input; and
output at least one of the first value, the second value, or the third value.

### Claim 28 (depends on 22)

The at least one non-transitory computer readable medium of claim 22, wherein the instructions, when executed, cause the first processor circuitry to:
in response to the triggering of the breakpoint, at least one of:
adjust a first value of the data input;
adjust a second value of a first register of a core of the first processor circuitry or the second processor circuitry; or
adjust a third value of a second register of the debug circuitry; and
resume the execution of the executable code based on at least one of the first value, the second value, or the third value.

### Claim 29

An apparatus to debug a hardware accelerator, the apparatus comprising:
first interface circuitry to obtain a machine-learning model; and
processor circuitry including one or more of:
at least one of a central processing unit, a graphics processing unit, or a digital signal processor, the at least one of the central processing unit, the graphics processing unit, or the digital signal processor having control circuitry to control data movement within the processor circuitry, arithmetic and logic circuitry to perform one or more first operations corresponding to instructions, and one or more registers to store a result of the one or more first operations, the instructions in the apparatus;
a Field Programmable Gate Array (FPGA), the FPGA including logic gate circuitry, a plurality of configurable interconnections, and storage circuitry, the logic gate circuitry and interconnections to perform one or more second operations, the storage circuitry to store a result of the one or more second operations; or
Application Specific Integrate Circuitry (ASIC) including logic gate circuitry to perform one or more third operations;
the processor circuitry to perform at least one of the first operations, the second operations, or the third operations to instantiate:
core circuitry to execute executable code to generate a data output based on a data input, the executable code based on the machine-learning model;
second interface circuitry to receive at least one of the data input or the data output;
multiplexer circuitry to select the second interface circuitry; and
a shift register to output the at least one of the data input or the data output in response to a trigger of a breakpoint associated with an execution of the executable code.

### Claim 30 (depends on 29)

The apparatus of claim 29, wherein the second interface circuitry is to receive the data input from memory, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate:
a buffer to, in response to the breakpoint not being triggered:
receive the data input from the second interface circuitry; and
output the data input to the core circuitry.

### Claim 31 (depends on 29)

The apparatus of claim 29, wherein the second interface circuitry is to receive the data output from the core circuitry, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate:
a buffer to, in response to the breakpoint not being triggered:
receive the data output from the second interface circuitry; and
output the data output to memory.

### Claim 32 (depends on 29)

The apparatus of claim 29, wherein the second interface circuitry is to receive the data input from memory, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate:
a buffer to, in response to the breakpoint not being triggered:
receive the data input from the second interface circuitry; and
output the data input to the core circuitry; and
debug circuitry to:
output the at least one of the data input or the data output in response to the trigger of the breakpoint; or
output the data output to the memory in response to the breakpoint not being triggered.

### Claim 33 (depends on 32)

The apparatus of claim 32, wherein the debug circuitry is included in the core circuitry.

### Claim 34 (depends on 29)

The apparatus of claim 29, wherein the at least one of the data input or the data output includes a first value, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate:
a configuration register to store a second value corresponding to the breakpoint; and
comparator circuitry to:
compare the first value and the second value; and
instruct the second interface circuitry to provide the at least one of the data input or the data output to the multiplexer circuitry in response to a match of the first value and the second value based on the comparison, the triggering of the breakpoint in response to the match.

### Claim 35 (depends on 29)

The apparatus of claim 29, wherein the core circuitry is first core circuitry, and the processor circuitry is to perform at least one of the first operations, the second operations, or the third operations to instantiate:
counter circuitry to:
output a first value to instruct the multiplexer circuitry to select the output of the second interface circuitry; and
output a second value to instruct the multiplexer circuitry to select an output of third interface circuitry associated with second core circuitry.

### Claim at position 36

36-42. (canceled)
