# US 2019/0377670 A1 · Tester and method for testing a device under test using relevance scores

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0377670 A1
- Document ID: US20190377670A1
- Status: Published application
- Kind: A1
- Application number: 16/548,653
- Filed: 2019-08-22
- Published: 2019-12-12
- Earliest claimed: 2017-03-07
- Assignees: Advantest Corporation
- Applicants: Advantest Corporation
- Inventors: Jochen RIVOIR
- Web page: https://patentgrep.com/patent/US20190377670A1

## CPC

- G06F 11/3692: {for test results analysis}
- H04L 43/50: Testing arrangements
- G01R 31/287: {Procedures; Software aspects}
- G06F 11/368: {for test version control, e.g. updating test cases to a new software version}
- G06F 11/3684: {for test design, e.g. generating new test cases}
- G06F 11/3688: {for test execution, e.g. scheduling of test suites}
- H04L 43/00: Arrangements for monitoring or testing data switching networks
- G05B 23/0256: {injecting test signals and analyzing monitored process response, e.g. injecting the test signal while interrupting the normal operation of the monitored system; superimposing the test signal onto a control signal during normal operation of the monitored system}
- G06F 2218/00: Aspects of pattern recognition specially adapted for signal processing
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G01R: MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES (indicating correct tuning of resonant circuits H03J3/12)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

A tester for testing a device under test is shown, having a test unit configured for performing a test of the device under test using multiple test cases, each test case having variable values of a set of predetermined variables, the test units configured to derive an output value for each test case indicating whether the device under test validly operates at a current test case or whether the device under test provides an error at the current test case; and an evaluation unit configured for evaluating the multiple test cases based on a plurality of subsets of the predetermined input variables with respect to the output value, the evaluation unit configured for providing a number of plots of the evaluation of the multiple test cases where each plot indicates the impact of one subset of the plurality of subsets of the predetermined input variables to the output value in dependence on respective relevance scores or associated with the respective relevance scores.

## Claims

### Claim 1

A tester for testing a device under test, the tester comprising:
a test unit configured for performing a test of the device under test using multiple test cases, wherein each test case of the multiple test cases comprises variable values of a set of predetermined input variables and wherein the test unit is configured to derive an output value for each test case, wherein the output value indicates if the device under test provides a valid response for a current test case or whether the device under test provides an error in response to the current test case; and
an evaluation unit configured to:
evaluate each of the multiple test cases based on a plurality of subsets of the set of predetermined input variables and a respective resultant output value, and wherein each of the subsets of the predetermined input variables are smaller than the set of predetermined input variables;
provide a number of plots associated with an evaluation of the multiple test cases, wherein each plot indicates an impact of one subset of the plurality of subsets of the predetermined input variables to the respective resultant output value of a test case using a relevance score; and
determine relevance scores for the plurality of subsets of the set of predetermined input variables, wherein each respective relevance score indicates a respective relevance of each of the plurality of subsets when determining an impact of the plurality of subsets of the set of predetermined input variables on errors of the device under test.

### Claim 2 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to determine the relevance scores using entropy measures, wherein the evaluation unit is further configured to determine the entropy measures using:
a conditional entropy of an error distribution resulting if a given subset of the plurality of subsets of the set of predetermined input variables comprises a number of predetermined input variables that is greater than 1; or
an entropy of an error distribution resulting if a given subset of the plurality of subsets of the set of predetermined input variables comprises a number of predetermined input variables that is is equal to 1.

### Claim 3 (depends on 2)

The tester according to claim 2, wherein the evaluation unit is configured to calculate believability measures, wherein the believability measures indicate a statistical uncertainty of an error distribution resulting from a dependence of a respective one of the plurality of subsets on predetermined input variables of a given subset of the plurality of subsets and wherein the evaluation unit is further configured to adjust the entropy measures using the believability measures to decrease the relevance scores of those subsets of the plurality of subsets of the set of predetermined input variables with an increasing statistical uncertainty and to increase the relevance score of those subsets of the plurality of subsets of set of the predetermined input variables with a decreasing statistical uncertainty.

### Claim 4 (depends on 3)

The tester according to claim 3, wherein the evaluation unit is further configured to calculate the believability measure using a confidence of deviation of the errors, wherein the confidence of deviation is calculated using a computation of a deviation that a probability between an empirical error probability of the given subset is greater or equal to an overall error probability and a probability that the empirical error probability of the given subset is smaller than the overall error probability.

### Claim 5 (depends on 3)

The tester according to claim 3, wherein the evaluation unit is further configured to calculate the believability measure using an average confidence of deviation, wherein the average confidence calculated as a mean of deviations between a probability that an empirical error probability is greater or equal to an overall error probability and a probability that the empirical error probability is smaller than the overall error probability in a range from the empirical error probability when determined in dependence on input variables of a given subset of the plurality of subsets of the set of predetermined input variables to the overall error probability.

### Claim 6 (depends on 3)

The tester according to claim 3, wherein the evaluation unit is further configured to calculate the believability measure using a variance of deviation based on a ratio of:
a deviation between a number of errors in subranges of values of input variables of a given subset of predetermined input variables and an expected number of errors in subranges of values of input variables of the set of predetermined input variables based on an overall error probability; and
a standard deviation of numbers of errors.

### Claim 7 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to determine the relevance scores using a geometrical measure of an error distribution within a value range covered by values of predetermined input variables in the plurality of subsets of the set of predetermined input variables.

### Claim 8 (depends on 7)

The tester according to claim 7, wherein the evaluation unit is further configured to determine the geometrical measure based on conditional error probabilities within subranges of the plurality of subsets of the set of predetermined input variables.

### Claim 9 (depends on 8)

The tester according to claim 8, wherein the evaluation unit is further configured to define a threshold value of a conditional error probability in the subranges that distinguishes between error-prone subranges and non-error-prone subranges and to increase the geometrical measure with an increasing number of distinct error-prone subranges.

### Claim 10 (depends on 8)

The tester according to claim 8, wherein the evaluation unit is further configured to define a threshold value of the conditional error probability in the subranges that distinguishes between error-prone subranges and non-error-prone subranges and to increase the geometrical measure with a decreasing size of groups of adjacent error-prone subranges surrounded by non-error-prone subranges.

### Claim 11 (depends on 8)

The tester according to claim 8, wherein the evaluation unit is further configured to define a threshold value of the conditional error probability in the subranges that distinguishes between error-prone subranges and non-error-prone subranges and to increase the geometrical measure with a decreasing distance of the error-prone subranges to minimum and/or maximum values of the subranges.

### Claim 12 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to calculate a relevance score using a weighted sum of an entropy measure and a geometrical measure.

### Claim 13 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to rank the number of plots according to the relevance scores and to provide the number of plots to a user of the tester based on the ranking.

### Claim 14 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to calculate a transformation variable using, for each test case, at least two variable values of input variables and to extend the set of input variables by the transformation variable, wherein an extended set of input variables is formed and wherein the evaluation unit is further configured to evaluate the multiple test cases based on the plurality of subsets of an extended set of predetermined input variables.

### Claim 15 (depends on 14)

The tester according to claim 14, wherein the evaluation unit is further configured to perform a coordinate transform, wherein a functional dependency of the set of predetermined input variables is achieved, and wherein variable values of the set of predetermined input variables are used to calculate the transformation variable.

### Claim 16 (depends on 14)

The tester according to claim 14, wherein a count of the plurality of subsets of the extended set of predetermined input variables is smaller than or equal to 4.

### Claim 17 (depends on 1)

The tester according to claim 1, wherein the evaluation unit is further configured to:
quantize the variable values of the set of predetermined input variables for each test case for evaluating the multiple test cases based on the plurality of subsets of the set of predetermined input variables with quantized variable values in relation to the output value, wherein one predetermined input variable represents one dimension of the plot and wherein quantization borders in each dimension define subranges of a value range covered by the variable values of the predetermined input variables;
count a total number of test cases situated in each subrange and to determine the number of test cases where an error occurred in each test case;
determine for each subrange an error probability that indicates a ratio of the number of test cases where an error occurred to the total number of test cases;
determine an entropy measure for each subset of the plurality of subsets of the set of predetermined input variables based on the total number of test cases in each subrange, a number of test cases where an error occurred in each subrange and a overall count of the multiple test cases; and
calculate the relevance score using the entropy measure.

### Claim 18 (depends on 17)

The tester according to claim 17, wherein the evaluation unit is further configured to determine a geometrical measure based on a distribution of errors resulting from a dependence on input variables of a given subset of a plurality of subsets of the set of predetermined input variables and to calculate the relevance score further using the geometrical measure.

### Claim 19

A method for testing a device under test, the method comprising:
performing a test of the device under test using multiple test cases, wherein each test case comprises variable values of predetermined input variables;
deriving an output value for each test case indicating whether the device under test validly operates or provides an error at a current test case;
evaluating the multiple test cases based on a plurality of subsets of the predetermined input variables in relation to a respective the output value, and wherein the subsets of predetermined input variables are smaller than the predetermined input variables;
providing a plurality of plots associated with an evaluation of the multiple test cases, wherein each plot of the plurality of plots indicates an impact of one subset of the plurality of subsets of the predetermined input variables to the respective output value using a relevance score; and
determining relevance scores for the plurality of subsets of predetermined input variables, wherein each respective relevance score indicates a respective relevance of each of the plurality of subsets when determining an impact of the plurality of subsets of predetermined input variables on errors of the device under test.

### Claim 20

A non-transitory digital storage medium having stored thereon a computer program for performing a method of testing a device under test, the method comprising:
performing a test of the device under test using multiple test cases, wherein each test case comprises variable values of predetermined input variables;
deriving an output value for each test case indicating whether the device under test validly operates or provides an error at a current test case;
evaluating the multiple test cases based on a plurality of subsets of the predetermined input variables in relation to a respective output value, and wherein the subsets of predetermined input variables are smaller than the predetermined input variables;
providing a plurality of plots associated with an evaluation of the multiple test cases, wherein each plot of the plurality of plots indicates an impact of one subset of the plurality of subsets of the predetermined input variables to the respective output value using a relevance score; and
determining relevance scores for the plurality of subsets of predetermined input variables, wherein each respective relevance score indicates a respective relevance of each of the plurality of subsets when determining an impact of the plurality of subsets of predetermined input variables on errors of the device under test.

### Claim 21 (depends on 8)

The tester according to claim 8, wherein the evaluation unit is further configured to define a threshold value of a conditional error probability in the subranges that distinguishes between error-prone subranges and non-error-prone subranges and to increase the geometrical measure with an increasing number of groups of adjacent error-prone subranges surrounded by non-error-prone subranges.
