# US 2025/0209352 A1 · Method and device for validating explainability methods for a machine learning system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0209352 A1
- Document ID: US20250209352A1
- Status: Published application
- Kind: A1
- Application number: 18/971,828
- Filed: 2024-12-06
- Published: 2025-06-26
- Earliest claimed: 2023-12-20
- Applicants: Robert Bosch GmbH
- Inventors: Andreas Steimer; Clint Sebastian; Jana Veser; Laura Beggel; Mehul Bansal; Thi Phuong Nhung Ngo
- Web page: https://patentgrep.com/patent/US20250209352A1

## CPC

- G06N 5/045: Explanation of inference; Explainable artificial intelligence [XAI]; Interpretable artificial intelligence
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for validating an attribution-based explainability method for a machine learning system. The method includes ascertaining synthetic data points using a generator according to a noise vector; ascertaining an output of the machine learning system by propagating the synthetic data point through the machine learning system and ascertaining an explanation output using the attribution-based explainability method for the ascertained output; ascertaining a score of the explanation output; optimizing the noise vector with regard to the score so that the score moves to the rear part of the distribution of scores; ascertaining further synthetic data points using a generator according to the optimized noise vector; ascertaining a further output of the machine learning system by propagating the further synthetic data points through the machine learning system and ascertaining a further explanation output using the attribution-based explainability method for the further ascertained output; and validating the attribution-based explainability method.

## Claims

### Claim 1

A method for validating an attribution-based explainability method for a machine learning system, the method comprising the following steps:
ascertaining synthetic data points using a generator according to specified noise vectors;
ascertaining outputs of the machine learning system by propagating the synthetic data points through the machine learning system and ascertaining explanation outputs using the attribution-based explainability method for the ascertained outputs;
ascertaining scores of the explanation outputs that characterize in which section the explanation outputs lie in a distribution of explainability scores;
optimizing one of the noise vectors with regard to the score in such a way that the score moves to a rear part of the distribution of explanability scores;
ascertaining a further synthetic data point using the generator according to the optimized noise vector;
ascertaining a further output of the machine learning system by propagating the further synthetic data point through the machine learning system and ascertaining a further explanation output using the attribution-based explainability method for the further ascertained output;
ascertaining a score of the further explanation output; and
validating the attribution-based explainability method, wherein when the score lies in a rear part of a distribution of an empirically ascertained distribution of explainability scores and the further synthetic data point does not lie in a rear part of a distribution of the synthetic data points, a positive validation is given.

### Claim 2 (depends on 1)

The method according to claim 1, wherein a rear part of the distribution is defined by a specified percentile.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the optimizing of the one of the noise vectors is carried out with a gradient-based or gradient-free optimization method.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the machine learning system is used for an optical inspection of produced components.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the synthetic data points are images and the machine learning system is an image classifier, wherein a technical system can be controlled according to classifications of the machine learning system.

### Claim 6

A device configured to validating an attribution-based explainability method for a machine learning system, the system configured to:
ascertain synthetic data points using a generator according to specified noise vectors;
ascertain outputs of the machine learning system by propagating the synthetic data points through the machine learning system and ascertaining explanation outputs using the attribution-based explainability method for the ascertained outputs;
ascertain scores of the explanation outputs that characterize in which section the explanation outputs lie in a distribution of explainability scores;
optimize one of the noise vectors with regard to the score in such a way that the score moves to a rear part of the distribution of explanability scores;
ascertain a further synthetic data point using the generator according to the optimized noise vector;
ascertain a further output of the machine learning system by propagating the further synthetic data point through the machine learning system and ascertaining a further explanation output using the attribution-based explainability method for the further ascertained output;
ascertain a score of the further explanation output; and
validate the attribution-based explainability method, wherein when the score lies in a rear part of a distribution of an empirically ascertained distribution of explainability scores and the further synthetic data point does not lie in a rear part of a distribution of the synthetic data points, a positive validation is given.

### Claim 7

A non-transitory machine-readable storage medium on which is stored a computer program for validating an attribution-based explainability method for a machine learning system, the computer program, when executed by a computer, causing the computer to perform the following steps:
ascertaining synthetic data points using a generator according to specified noise vectors;
ascertaining outputs of the machine learning system by propagating the synthetic data points through the machine learning system and ascertaining explanation outputs using the attribution-based explainability method for the ascertained outputs;
ascertaining scores of the explanation outputs that characterize in which section the explanation outputs lie in a distribution of explainability scores;
optimizing one of the noise vectors with regard to the score in such a way that the score moves to a rear part of the distribution of explanability scores;
ascertaining a further synthetic data point using the generator according to the optimized noise vector;
ascertaining a further output of the machine learning system by propagating the further synthetic data point through the machine learning system and ascertaining a further explanation output using the attribution-based explainability method for the further ascertained output;
ascertaining a score of the further explanation output; and
validating the attribution-based explainability method, wherein when the score lies in a rear part of a distribution of an empirically ascertained distribution of explainability scores and the further synthetic data point does not lie in a rear part of a distribution of the synthetic data points, a positive validation is given.