# US 11,907,098 B2 · Method for measuring performance of neural processing device and device for measuring performance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,098 B2 - Document ID: US11907098B2 - Status: Granted - Kind: B2 - Application number: 18/191,731 - Filed: 2023-03-28 - Published: 2023-10-05 - Granted: 2024-02-20 - Earliest claimed: 2022-04-01 - Assignees: Rebellions Inc. - Applicants: Rebellions Inc. - Inventors: Jinseok Kim - Web page: https://patentgrep.com/patent/US11907098B2 ## CPC - G06F 11/3495: {for systems} - G06F 11/3404: {for parallel or distributed programming} - G06F 11/3409: {for performance assessment} - G06F 11/3447: {Performance evaluation by modeling} - G06F 15/167: using a common memory, e.g. mailbox - G06F 15/781: {On-chip cache; Off-chip memory} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for measuring performance of neural processing devices and devices for measuring performance are provided. The method for measuring performance of neural processing devices comprises receiving hardware information of a neural processing device, modeling hardware components according to the hardware information as agents, dividing a calculation task by events for the agents and modeling the calculation task, thereby generating an event model which includes nodes corresponding to the agents and edges corresponding to the events and measuring a total duration of the calculation task through simulation of the event model. ## Claims ### Claim 1 A method for measuring performance of a neural processing device, comprising: receiving, at a processor, hardware information associated with the neural processing device; modeling, via the processor, hardware components as agents, according to the hardware information; generating, via the processor an event model by dividing a calculation task by events for the agents and modeling the calculation task, wherein the event model includes nodes corresponding to the agents and edges corresponding to the events; and measuring, via the processor, a total duration of the calculation task through simulation of the event model based on durations of the edges, each of the edges is data transmission between nodes, wherein at least one duration of the durations of the edges is updated by removing an edge. ### Claim 2 (depends on 1) The method for measuring performance of the neural processing device of claim 1, wherein generating the event model further comprises the processor performing operations including: generating a pre-event model that comprises at least one undetermined duration, wherein the nodes and the edges are formed in the pre-event model, and wherein each of the at least one undetermined duration comprises a fixed duration determined according to the hardware information and a bandwidth duration determined by at least one bandwidth; computing the fixed duration; estimating the bandwidth duration; determining a value of each of the at least one undetermined duration through the fixed duration and the bandwidth duration; and converting each of the at least one undetermined duration into a final duration. ### Claim 3 (depends on 2) The method for measuring performance of the neural processing device of claim 2, wherein estimating the bandwidth duration comprises the processor performing operations including: distributing a value of a total read bandwidth of each of the nodes evenly to a value of a read bound for each of the edges departing from each of the nodes; distributing a value of a total write bandwidth for each of the nodes evenly to a value of a write bound for each of the edges arriving at each of the nodes; determining a minimum value, as a final bandwidth, for each of the edges out of the read bound, the write bound, and an individual bound, determined according to the hardware information; and estimating the bandwidth duration according to the determined final bandwidth. ### Claim 4 (depends on 3) The method for measuring performance of the neural processing device of claim 3, wherein determining the minimum value as the final bandwidth further comprises the processor performing operations including: determining the individual bound as the final bandwidth of at least one of the edges whose individual bound is smaller than the read bound and the write bound; and resetting the read bound and the write bound evenly except for the at least one of the edges whose final bandwidth has been determined. ### Claim 5 (depends on 4) The method for measuring performance of the neural processing device of claim 4, wherein determining the minimum value as the final bandwidth further comprises the processor performing operations including: if there is no edge whose individual bound is smaller than the read bound and the write bound and if the read bound of each of all the edges departing from one node is less than or equal to the write bound of each of all the edges, determining the read bound of each of the edges as the final bandwidth of each of the edges; and resetting the write bound evenly, except for edges whose final bandwidth has been determined. ### Claim 6 (depends on 4) The method for measuring performance of the neural processing device of claim 4, wherein determining the minimum value as the final bandwidth further comprises the processor performing operations including: if there is no edge whose individual bound is smaller than the read bound and the write bound and if the write bound of each of all the edges arriving at one node is less than or equal to the read bound of each of all the edges, determining the write bound of each of the edges as the final bandwidth of each of the edges; and resetting the read bound evenly except for edges whose final bandwidth has been determined. ### Claim 7 (depends on 2) The method for measuring performance of the neural processing device of claim 2, wherein the event comprises: a compute job; and a memory job, wherein a duration of the compute job is determined according to the hardware information, wherein a duration of the memory job comprises an initial latency determined according to the hardware information plus the bandwidth duration determined dynamically according to a final bandwidth, and wherein the fixed duration comprises the duration of the compute job and the initial latency. ### Claim 8 (depends on 1) The method for measuring performance of the neural processing device of claim 1, wherein measuring the total duration comprises the processor performing operations including: advancing time by a final duration of an edge that is completed earliest among the edges; removing the edge that is completed the earliest; computing a final duration of each of all the edges again; and measuring the total duration by summing the advanced time if all the edges have been removed. ### Claim 9 (depends on 1) The method for measuring performance of the neural processing device of claim 1, wherein the hardware information comprises at least one of a clock frequency, a memory size, an initial latency that exists for data transfer between memories independent of data amount, a total bandwidth including an overall read bandwidth and an overall write bandwidth of a memory, a maximum multiple outstanding indicating a maximum number of requests that one hardware element can transmit, or a maximum bandwidth of a hardware component. ### Claim 10 (depends on 1) The method for measuring performance of the neural processing device of claim 1, wherein the hardware information comprises at least one of system specifications or design specifications before implementation of the neural processing device. ### Claim 11 A device for measuring performance, comprising processing circuitry, comprising: a hardware modeling module configured to: receive hardware information associated with a neural processing device; and generate a hardware model by modeling hardware components as agents; an event modeling module configured to: receive the hardware information and a calculation task; and generate an event model by dividing the calculation task by events for the agents based on the hardware model, wherein the event model includes nodes corresponding to the agents and edges corresponding to the events; and an event simulation module configured to measure performance of the neural processing device by simulating the event model based on durations of the edges, each of the edges is data transmission between nodes, wherein at least one duration of the durations of the edges is updated by removing an edge. ### Claim 12 (depends on 11) The device for measuring performance of claim 11, wherein the agents comprise: core agents for processing units included in the neural processing device; and memory agents for memories included in the neural processing device. ### Claim 13 (depends on 12) The device for measuring performance of claim 12, wherein each of the events comprises: a compute job performed by the core agents; and a memory job according to data transfer between the memory agents. ### Claim 14 (depends on 12) The device for measuring performance of claim 12, wherein each of the memory agents comprises: at least one load unit agent; and at least one store unit agent, wherein the load unit agent and the store unit agent are present in each of the memories. ### Claim 15 (depends on 11) The device for measuring performance of claim 11, wherein the event modeling module comprises: an event model generating module configured to generate a first pre-event model according to the hardware model by dividing the calculation task by the events, wherein the nodes and the edges are formed in the first pre-event model, and wherein durations of the edges are undetermined; a calculation module configured to generate a second pre-event model by computing fixed values out of the durations of the first pre-event model according to the hardware information; a bandwidth estimation module configured to generate a third pre-event model by determining bandwidth durations of the second pre-event model; and a duration decision module configured to receive the third pre-event model to determine final durations and generate the event model. ### Claim 16 (depends on 15) The device for measuring performance of claim 15, wherein the event simulation module advances time by one of the final durations and each of the final durations is associated with each of the edges, and wherein, when the final duration of an edge is completed the earliest among the edges of the event model, the event simulation module removes the edge whose final duration is completed the earliest and computes the final durations of each of the edges again. ### Claim 17 (depends on 11) The device for measuring performance of claim 11, wherein the neural processing device comprises: a central processing unit (CPU) configured to perform a first calculation; a neural core system-on-a-chip (SoC) configured to perform a second calculation that is different from the first calculation; and an off-chip memory configured to store data of the CPU and the neural core SoC. ### Claim 18 (depends on 17) The device for measuring performance of claim 17, wherein the neural core SoC comprises: at least one neural processor; a shared memory that is shared by the at least one neural processor; and a global interconnection configured to transmit data between the at least one neural processor and the shared memory. ### Claim 19 (depends on 18) The device for measuring performance of claim 18, wherein the global interconnection comprises: a data channel configured to communicate data; a control channel configured to communicate control signals; and a level 3 synchronization (L3 sync) channel configured to communicate synchronization signals. ### Claim 20 (depends on 19) The device for measuring performance of claim 19, wherein the event simulation module measures a total duration by accounting for synchronization durations.