# US 2024/0193472 A1 · Configuring an explanatory model to output consistent explanations across inputs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0193472 A1
- Document ID: US20240193472A1
- Status: Published application
- Kind: A1
- Application number: 18/080,290
- Filed: 2022-12-13
- Published: 2024-06-13
- Earliest claimed: 2022-12-13
- Assignees: Red Hat, Inc.
- Applicants: Red Hat, Inc.
- Inventors: Robert Joao Geada; Rui Miguel Cardoso De Freitas Machado Vieira
- Web page: https://patentgrep.com/patent/US20240193472A1

## CPC

- G06N 20/00: Machine learning
- G06N 3/045: Combinations of networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

In some examples, a system can receive input datasets that each include a respective set of variable values for the same set of variables, where the input datasets have different values for a particular variable in the set of variables. The system can generate encoded input datasets by encoding the different values of the particular variable. The system can also generate encoded background data. The system can then generate synthetic data based on the encoded input datasets and the encoded background data. The system can provide the synthetic data as input to a target model to receive outputs from the target model. The system can then configure an explanatory model based on the synthetic data and the outputs from the target model. The explanatory model can generate scores indicating how a selected set of values for the set of variables influences an output from the target model.

## Claims

### Claim 1

A non-transitory computer-readable medium comprising program code that is executable by one or more processors for causing the one or more processors to:
receive input datasets that each include a respective set of variable values for a same set of variables, wherein the input datasets have different values for a particular variable in the set of variables;
generate encoded input datasets by performing one-hot encoding with respect to the different values of the particular variable;
generate encoded background data based on how many different values of the particular variable are present in the input datasets;
generate synthetic data based on the encoded input datasets and the encoded background data, wherein the synthetic data includes multiple permutations of the encoded input datasets and the encoded background data;
provide the synthetic data as input to a target model to receive outputs from the target model; and
configure an explanatory model based on the synthetic data and the outputs from the target model, the explanatory model being configured to generate scores indicating how a selected set of values for the set of variables influences an output from the target model.

### Claim 2 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein each of the scores indicates how a respective value in the selected set of values influenced the output from the target model.

### Claim 3 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the explanatory model is a Shapley additive explanations (SHAP) model.

### Claim 4 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the explanatory model is separate from the target model.

### Claim 5 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the target model is a machine-learning model trained on training data that is different from the synthetic data.

### Claim 6 (depends on 1)

The non-transitory computer-readable medium of claim 1, further comprising program code that is executable by the one or more processors for causing the one or more processors to configure the explanatory model based on the synthetic data by:
fitting the explanatory model to the synthetic data.

### Claim 7 (depends on 1)

The non-transitory computer-readable medium of claim 1, wherein the input datasets includes a first input dataset and a second input dataset, and further comprising program code that is executable by the one or more processors for causing the one or more processors to generate the encoded input datasets by:
determining a first value of the particular variable in the first input dataset;
determining a second value of the particular variable in the second input dataset, the second value being different from the first value;
generating a first encoded dataset that includes a one-hot encoded representation of the first value; and
generating a second encoded dataset that includes a one-hot encoded representation of the second value, wherein the first encoded dataset and the second encoded dataset are included in the encoded input datasets.

### Claim 8

A method comprising:
receiving, by one or more processors, input datasets that each include a respective set of variable values for a same set of variables, wherein the input datasets have different values for a particular variable in the set of variables;
generating, by the one or more processors, encoded input datasets by performing one-hot encoding with respect to the different values of the particular variable;
generating, by the one or more processors, encoded background data based on how many different values of the particular variable are present in the input datasets;
generating, by the one or more processors, synthetic data based on the encoded input datasets and the encoded background data, wherein the synthetic data includes multiple permutations of the encoded input datasets and the encoded background data;
providing, by the one or more processors, the synthetic data as input to a target model to receive outputs from the target model; and
configuring, by the one or more processors, an explanatory model based on the synthetic data and the outputs from the target model, the explanatory model being configured to generate scores indicating how a selected set of values for the set of variables influences an output from the target model.

### Claim 9 (depends on 8)

The method of claim 8, wherein each of the scores indicates how a respective value in the selected set of values influenced the output from the target model.

### Claim 10 (depends on 8)

The method of claim 8, wherein the explanatory model is a Shapley additive explanations (SHAP) model.

### Claim 11 (depends on 8)

The method of claim 8, wherein the explanatory model is separate from the target model.

### Claim 12 (depends on 8)

The method of claim 8, wherein the target model is a machine-learning model trained on training data that is different from the synthetic data.

### Claim 13 (depends on 8)

The method of claim 8, further comprising configuring the explanatory model based on the synthetic data by:
fitting the explanatory model to the synthetic data.

### Claim 14 (depends on 8)

The method of claim 8, wherein the input datasets includes a first input dataset and a second input dataset, and further comprising generating the encoded input datasets by:
determining a first value of the particular variable in the first input dataset;
determining a second value of the particular variable in the second input dataset, the second value being different from the first value;
generating a first encoded dataset that includes a one-hot encoded representation of the first value; and
generating a second encoded dataset that includes a one-hot encoded representation of the second value, wherein the first encoded dataset and the second encoded dataset are included in the encoded input datasets.

### Claim 15

A system comprising:
one or more processors; and
one or more memories includes instructions that are executable by the one or more processors for causing the one or more processors to:
receive input datasets that each include a respective set of variable values for a same set of variables, wherein the input datasets have different values for a particular variable in the set of variables;
generate encoded input datasets by performing one-hot encoding with respect to the different values of the particular variable;
generate encoded background data based on how many different values of the particular variable are present in the input datasets;
generate synthetic data based on the encoded input datasets and the encoded background data, wherein the synthetic data includes multiple permutations of the encoded input datasets and the encoded background data;
provide the synthetic data as input to a target model to receive outputs from the target model; and
configure an explanatory model based on the synthetic data and the outputs from the target model, the explanatory model being configured to generate scores indicating how a selected set of values for the set of variables influences an output from the target model.

### Claim 16 (depends on 15)

The system of claim 15, wherein each of the scores indicates how a respective value in the selected set of values influenced the output from the target model.

### Claim 17 (depends on 15)

The system of claim 15, wherein the explanatory model is a Shapley additive explanations (SHAP) model that is separate from the target model.

### Claim 18 (depends on 15)

The system of claim 15, wherein the target model is a machine-learning model trained on training data that is different from the synthetic data.

### Claim 19 (depends on 15)

The system of claim 15, wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to configure the explanatory model based on the synthetic data by:
fitting the explanatory model to the synthetic data.

### Claim 20 (depends on 15)

The system of claim 15, wherein the input datasets includes a first input dataset and a second input dataset, and wherein the one or more memories further include instructions that are executable by the one or more processors for causing the one or more processors to generate the encoded input datasets by:
determining a first value of the particular variable in the first input dataset;
determining a second value of the particular variable in the second input dataset, the second value being different from the first value;
generating a first encoded dataset that includes a one-hot encoded representation of the first value; and
generating a second encoded dataset that includes a one-hot encoded representation of the second value, wherein the first encoded dataset and the second encoded dataset are included in the encoded input datasets.
