# US 10,769,331 B2 · Verification algorithm engine selection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,769,331 B2
- Document ID: US10769331B2
- Status: Granted
- Kind: B2
- Application number: 16/033,417
- Filed: 2018-07-12
- Published: 2020-01-16
- Granted: 2020-09-08
- Earliest claimed: 2018-07-12
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Peng Fei Gou; Heng Liu; Yang Fan Liu; Yan Heng Lu; Chen Qian; Zhen Peng Zuo
- Web page: https://patentgrep.com/patent/US10769331B2

## CPC

- G06F 30/33: Design verification, e.g. functional simulation or model checking
- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06F 30/327: Logic synthesis; Behaviour synthesis, e.g. mapping logic, HDL to netlist, high-level language to RTL or netlist
- G06F 30/3323: using formal methods, e.g. equivalence checking or property checking
- G06F 30/337: Design optimisation
- G06F 30/30: Circuit design
- G06F 30/367: Design verification, e.g. using simulation, simulation program with integrated circuit emphasis [SPICE], direct methods or relaxation methods
- G06F 30/373: Design optimisation
- G06F 30/398: Design verification or optimisation, e.g. using design rule check [DRC], layout versus schematics [LVS] or finite element methods [FEM] (optical proximity correction [OPC] design processes G03F1/36)
- G06F 2111/02: CAD in a network environment, e.g. collaborative CAD or distributed simulation
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

An apparatus for verification includes a processing module, a data collection module, an engine selection module and an engine execution module. The processing module processes a netlist using a plurality of engines. The netlist includes components and nodes of an integrated circuit design. Each engine includes an algorithm for verification of the integrated circuit design. The data collection module stores, for each engine, execution results for the engine for a plurality of netlists, the results stored in a history buffer. The engine selection module, for a current netlist, calculates using execution results in the history buffer which engine of the plurality of engines has a highest predicted performance and selects the engine with the highest predicted performance. The engine execution module executes the current netlist using the selected engine to produce execution results, reports the execution results and stores the execution results in the history buffer.

## Claims

### Claim 1

An apparatus comprising:
a processing module that processes a netlist using a plurality of engines, the netlist comprising components and nodes of an integrated circuit design, each engine comprising an algorithm for verification of the integrated circuit design, wherein at least a portion of the plurality of engines comprise a transformation-based verification (“TBV”) engine that simplifies the netlist by reducing redundant elements;
a data collection module that stores, for each engine, execution results for the engine for a plurality of netlists, the results stored in a history buffer;
an engine selection module that, for a current netlist, calculates using execution results in the history buffer which engine of the plurality of engines has a highest predicted performance and selects the engine with the highest predicted performance; and
an engine execution module that executes the current netlist using the selected engine to produce execution results, reports the execution results, and stores the execution results in the history buffer,
wherein said modules comprise one or more of hardware circuits, a programmable hardware device and executable code, the executable code stored on one or more computer readable storage media.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein execution results of the engine with the highest predicted performance comprises a modified netlist and further comprising a further processing module that uses, one or more times, a modified netlist from a previous engine execution, the engine selection module and the engine execution module to select an engine with a highest predicted performance for the modified netlist, executes the modified netlist with the selected engine, reports the execution results and stores the execution results to the history buffer.

### Claim 3 (depends on 2)

The apparatus of claim 2, wherein one or more of the plurality of engines one or more of:
reduce a quantity of one or more parameters of the netlist; and
search possible conditions within the netlist and provide one of:
notification of a design error;
validation of the integrated circuit design; and
notification that validation of the integrated circuit design is incomplete.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein the further processing module terminates engine selection and execution in response to the engine execution module reporting execution results that indicate a validation of the integrated circuit design.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the data collection module further comprises an engine performance module that, for each engine, compiles in the history buffer engine output values from netlist execution results of an engine in an ordered list, each ordered list comprising one or more of an engine identifier, an engine output value for each output of the engine and a relationship, the relationship comprising how the engine used for execution relates to one or more other engines.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the engine selection module further comprises an engine normalization module that, for the current netlist, averages the engine output values for one or more engines of the plurality of engines and wherein the engine selection module selects an engine with a highest average as the engine with the highest predicted performance.

### Claim 7 (depends on 6)

The apparatus of claim 6, wherein the engine normalization module averages each engine output value by normalizing each engine output value using a deduction ratio based on the engine output value of a netlist parameter of the execution results and an input value of the same netlist parameter from the current netlist used as input for the engine being executed.

### Claim 8 (depends on 6)

The apparatus of claim 6, wherein the engine selection module further comprises a machine learning module that modifies the normalized engine output values based on a machine learning inference algorithm and execution results in the history buffer, wherein the machine learning inference algorithm modifies the normalized engine output values based on one or more of:
properties of netlists used as input for execution results in the history buffer;
properties of the current netlist;
available computing resources; and
execution time limits.

### Claim 9 (depends on 5)

The apparatus of claim 5, wherein the engine output values comprise one or more of a number of registers, a number of gates, a number of inputs, a number of arrays, a number of constraints, execution time and memory used during execution.

### Claim 10

A computer-implemented method for verification comprising:
processing a netlist using a plurality of engines, the netlist comprising components and nodes of an integrated circuit design, each engine comprising an algorithm for verification of the integrated circuit design, wherein at least a portion of the plurality of engines comprise a transformation-based verification (“TBV”) engine that simplifies the netlist by reducing redundant elements;
storing, for each engine, execution results for the engine for a plurality of netlists, the results stored in a history buffer;
calculating, for a current netlist and using execution results in the history buffer, which engine of the plurality of engines has a highest predicted performance;
selecting the engine with the highest predicted performance;
executing the current netlist using the selected engine to produce execution results;
reporting the execution results; and
storing the execution results in the history buffer.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein execution results of the engine with the highest predicted performance comprise a modified netlist and further comprising, one or more times,
selecting an engine with a highest predicted performance for the modified netlist;
executing the selected engine on the modified netlist;
reporting the execution results in the history buffer; and
storing the associated execution results in the history buffer.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein one or more of the engines used for processing a netlist
reduce a quantity of one or more parameters of the netlist; and
search possible conditions within the netlist and provide one of:
notification of a design error;
validation of the integrated circuit design; and
notification that validation of the integrated circuit design is incomplete.

### Claim 13 (depends on 11)

The computer-implemented method of claim 11, further comprising terminating engine selection and execution in response to execution results comprising a validation of the integrated circuit design.

### Claim 14 (depends on 10)

The computer-implemented method of claim 10, wherein storing execution results further comprises compiling in the history buffer, for each engine, output values from netlist execution results of an engine in an ordered list, each ordered list comprising one or more of an engine identifier, an engine output value for each output of the engine and a relationship, the relationship comprising how the engine used for execution relates to one or more other engines.

### Claim 15 (depends on 14)

The computer-implemented method of claim 14, wherein selecting an engine with a highest predicted performance further comprises averaging, for the current netlist, the engine output values for one or more engines of the plurality of engines and selecting an engine with a highest average as the engine with the highest predicted performance.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein averaging each engine output value comprises normalizing each engine output value using a deduction ratio based on the engine output value of a netlist parameter of the execution results and an input value of the same netlist parameter from the current netlist used as input for the engine being executed.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, further comprises modifying the normalized engine output values based on a machine learning inference algorithm and execution results in the history buffer, wherein the machine learning inference algorithm modifies the normalized engine output values based on one or more of:
properties of netlists used as input for execution results in the history buffer;
properties of the current netlist;
available computing resources; and
execution time limits.

### Claim 18

A computer program product for verification, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
process a netlist using a plurality of engines, the netlist comprising components and nodes of an integrated circuit design, each engine comprising an algorithm for verification of the integrated circuit design, wherein at least a portion of the plurality of engines comprise a transformation-based verification (“TBV”) engine that simplifies the netlist by reducing redundant elements;
store execution results for the engine for a plurality of netlists, the results stored in a history buffer;
calculate, for a current netlist and using execution results in the history buffer, which engine of the plurality of engines has a highest predicted performance;
select the engine with the highest predicted performance;
execute the current netlist using the selected engine to produce execution results;
report the execution results; and
store the execution results in the history buffer.

### Claim 19 (depends on 18)

The computer program product of claim 18, wherein execution results of the engine with the highest predicted performance comprises a modified netlist and the program instructions further cause the processor to, one or more times,
select an engine with a highest predicted performance for the modified netlist;
execute the selected engine on the modified netlist;
report the execution results in the history buffer; and
store the associated execution results in the history buffer.

### Claim 20 (depends on 18)

The computer program product of claim 18, wherein the program instructions causing the processor to store execution results further comprise program instructions cause the processor to
compile in the history buffer, for each engine, output values from netlist execution results of an engine in an ordered list, each ordered list comprising one or more of an engine identifier, an engine output value for each output of the engine and a relationship, the relationship comprising how the engine used for execution relates to one or more other engines;
select an engine with a highest predicted performance further comprises program instructions causing the processor to average, for the current netlist, the engine output values for one or more engines of the plurality of engines and to select an engine with a highest average as the engine with the highest predicted performance wherein averaging each engine output value comprises normalizing each engine output value using a ratio of the engine output value to a common engine output value; and
modify the normalized engine output values based on a machine learning inference algorithm and execution results in the history buffer, wherein the machine learning inference algorithm modifies the normalized engine output values based on one or more of:
properties of netlists used as input for execution results in the history buffer;
properties of the current netlist;
available computing resources; and
execution time limits.