# US 12,111,740 B2 · System level testing of artificial intelligence primitives

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,111,740 B2
- Document ID: US12111740B2
- Status: Granted
- Kind: B2
- Application number: 18/059,633
- Filed: 2022-11-29
- Published: 2024-05-30
- Granted: 2024-10-08
- Earliest claimed: 2022-11-29
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Ali Y. Duale; Timothy Slegel; Patrick Duffy
- Web page: https://patentgrep.com/patent/US12111740B2

## CPC

- G06F 11/263: Generation of test inputs, e.g. test vectors, patterns or sequences {; with adaptation of the tested hardware for testability with external testers}
- G06F 11/261: {by simulating additional hardware, e.g. fault simulation}
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Methods, system, and computer program product are disclosed for enhanced system level testing of an AI hardware accelerator comprising Artificial Intelligence (AI) primitives. The system level testing provides uncompromised test coverage of the AI hardware accelerator while enabling efficient and effective testing performance. Enhanced test data generation, result prediction, and test execution are provided for system level testing of AI hardware accelerator including the AI primitives.

## Claims

### Claim 1

A computer implemented method comprising:
providing a set of input data;
replicating the set of input data to produce multiple units of input data;
processing each of the multiple units using an Artificial Intelligence (AI) hardware accelerator to provide a first output;
processing the set of input data using a software machine learning model to produce a result prediction output that is smaller than the multiple units of input data processed by the AI hardware accelerator;
replicating the result prediction output to generate an expected result output which is a same size as the first output; and
comparing the first output and the expected result output to validate the AI hardware accelerator.

### Claim 2 (depends on 1)

The method of claim 1, wherein the AI hardware accelerator comprises multiple function codes, and wherein providing a set of input data comprises providing test input data to satisfy a threshold testing requirement for one function code.

### Claim 3 (depends on 1)

The method of claim 1, wherein replicating the set of input data to produce multiple units of input data comprises replicating the set of input data to produce multiple pages of input data.

### Claim 4 (depends on 1)

The method of claim 1, wherein processing the set of input data using a software machine learning model to produce a result prediction output comprises producing a result prediction output of a single unit of input data.

### Claim 5 (depends on 1)

The method of claim 1, wherein comparing the first output and the expected result output to validate the AI hardware accelerator comprises identifying an error condition where the first output and the expected result output mismatch.

### Claim 6 (depends on 1)

The method of claim 1, further comprises configuring a daisy chain of multiple function codes comprising AI primitives of the AI hardware accelerator.

### Claim 7 (depends on 6)

The method of claim 6, further comprises processing test input data using the daisy chain of multiple function codes by the AI hardware accelerator to generate a respective output of each of the multiple function codes of the AI hardware accelerator.

### Claim 8 (depends on 6)

The method of claim 6, further comprises processing test input data using the daisy chain of multiple function codes by the AI hardware accelerator in multiple runs, and comparing generated outputs of the multiple runs to validate the AI primitives of the AI hardware accelerator.

### Claim 9 (depends on 8)

The method of claim 8, wherein comparing generated outputs of the multiple runs to validate the AI primitives of the AI hardware accelerator comprises identifying an error condition where the generated outputs of the multiple runs mismatch.

### Claim 10 (depends on 1)

The method of claim 1, wherein processing the set of input data using a software machine learning model to produce a result prediction output enables reduction of result prediction time for system level testing of an AI hardware accelerator.

### Claim 11

A system, comprising:
a processor; and
a memory, wherein the memory includes a computer program product to perform operations for testing of Artificial Intelligence (AI) hardware accelerator, the operations comprising:
providing a set of input data;
replicating the set of input data to produce multiple units of input data;
processing each of the multiple units using the AI hardware accelerator to provide a first output;
processing the set of input data using a software machine learning model to produce a result prediction output that is smaller than the multiple units of input data processed by the AI hardware accelerator;
replicating the result prediction output to generate an expected result output which is a same size as the first output; and
comparing the first output and the expected result output to validate the Al hardware accelerator.

### Claim 12 (depends on 11)

The system of claim 11, wherein the AI hardware accelerator comprises multiple function codes, and wherein providing a set of input data comprises providing valid input data to satisfy a threshold testing requirement for one function code.

### Claim 13 (depends on 11)

The system of claim 11, wherein replicating the set of input data to produce multiple units of input data comprises replicating the set of input data to produce multiple pages of input data.

### Claim 14 (depends on 11)

The system of claim 11, wherein processing the set of input data using a software machine learning model to produce a result prediction output comprises producing a result prediction output of a single unit of input data.

### Claim 15 (depends on 11)

The system of claim 11, wherein comparing the first output and the expected result output to validate the AI hardware accelerator comprises identifying an error condition where the first output and the expected result output mismatch.

### Claim 16

A computer program product for testing an Artificial Intelligence (AI) hardware accelerator, the computer program product comprising:
a non-transitory computer-readable storage medium having computer-readable program code embodied therewith, the computer-readable program code executable by one or more computer processors to perform an operation comprising:
providing a set of input data;
replicating the set of input data to produce multiple units of input data;
processing each of the multiple units using an AI hardware accelerator to provide a first output;
processing the set of input data using a software machine learning model to produce a result prediction output that is smaller than the multiple units of input data processed by the AI hardware accelerator;
replicating the result prediction output to generate an expected result output which is a same size as the first output; and
comparing the first output and the expected result output to validate the AI hardware accelerator.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the AI hardware accelerator comprises multiple function codes, and wherein providing a set of input data comprises providing valid input data to satisfy a threshold testing requirement for one function code.

### Claim 18 (depends on 16)

The computer program product of claim 16, wherein replicating the set of input data to produce multiple units of input data comprises replicating the set of input data to produce multiple pages of input data.

### Claim 19 (depends on 16)

The computer program product of claim 16, wherein processing the set of input data using a software machine learning model to produce a result prediction output comprises producing a result prediction output of a single unit of input data.

### Claim 20 (depends on 16)

The computer program product of claim 16, further comprises configuring a daisy chain of multiple function codes comprising AI primitives of the AI hardware accelerator, and processing test input data using the daisy chain of multiple function codes by the AI hardware accelerator to generate a respective output of each of the multiple function codes of the AI hardware accelerator.