# US 12,217,139 B2 · Transforming a trained artificial intelligence model into a trustworthy artificial intelligence model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,139 B2
- Document ID: US12217139B2
- Status: Granted
- Kind: B2
- Application number: 17/524,204
- Filed: 2021-11-11
- Published: 2022-03-03
- Granted: 2025-02-04
- Earliest claimed: 2019-10-15
- Assignees: Siemens Aktiengesellschaft
- Applicants: Siemens Aktiengesellschaft
- Inventors: Florian Büttner; Christian Tomani
- Web page: https://patentgrep.com/patent/US12217139B2

## CPC

- A61B 5/0035: {adapted for acquisition of images from more than one imaging mode, e.g. combining MRI and optical tomography}
- G06N 20/00: Machine learning
- A61B 5/7267: {involving training the classification device}
- G06F 18/2193: {based on specific statistical tests}
- G06F 21/6218: {to a system of files or objects, e.g. local or distributed file system or database}
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G06N 3/045: Combinations of networks
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The following relates to a computer-implemented method and system for transforming a trained artificial intelligence model into a trustworthy artificial intelligence model, by providing the trained artificial intelligence model via a user interface of a webservice platform, providing a validation data set, which is based on training data of the trained artificial intelligence model, generating generic samples by a computing component of the webservice platform based on the validation data set, and transforming the trained artificial intelligence model by optimizing a calibration based on the generic samples. The transformation of the AI model is performed by a computing component of the web service platform. The input, i.e. the trained artificial intelligence model as well as a validation data set, is provided to the computing component via a user interface of the web service platform. Such a user interface is implemented by any applicable frontend.

## Claims

### Claim 1

A computer-implemented method for transforming a trained artificial intelligence model into a trustworthy artificial intelligence model, comprising:
providing the trained artificial intelligence model via a user interface of a webservice platform, providing a validation data set, which is based on training data of the trained artificial intelligence model,
generating generic samples by a computing component of the webservice platform based on the validation data set, wherein for generating the generic samples the validation data set is modified by a domain-drift, and
transforming the trained artificial intelligence model by optimizing a calibration based on the generic samples, wherein by optimizing the calibration, an uncertainty-awareness is represented in a confidence-level for any of the generic samples.

### Claim 2 (depends on 1)

The method according to claim 1, wherein for generating the generic samples the validation data set is modified according to perturbation strengths.

### Claim 3 (depends on 1)

The method according to claim 1, wherein transforming comprises performing a re-training of the AI-model with applying an entropy-based loss term which encourages uncertainty-awareness.

### Claim 4 (depends on 3)

The method according to claim 3, wherein transforming further comprises applying a calibration loss term.

### Claim 5 (depends on 3)

The method according to claim 3, comprising the steps of:
generating current outputs of the AI model for the validation data set by forward propagating validation input data of the validation data set in the AI model;
computing a categorical cross-entropy loss LCCE for the validation data set based on the current outputs and corresponding ground truth data of the validation data set;
computing a predictive entropy loss LS by removing non-misleading evidence from the current outputs and distributing the remaining current outputs over a predetermined number C of classes;
computing a combined loss L by adding to the categorical cross-entropy loss LCCE the predictive entropy loss LS weighted with a predetermined first loss factor λS, where 0<=λS<=1;
checking whether the re-training converged to a predefined lower limit for a convergence rate;
updating weights of the AI model based on the combined loss L and a predetermined training rate η, where 0<η<=1, in case the re-training did not converge; and
stopping the re-training of the AI model in case the re-training converged.

### Claim 6 (depends on 5)

The method according to claim 5, further comprising the steps of:
generating perturbed outputs of the AI model for the generic samples by forward propagating the generic input data Xadv of the generic samples in the AI model;
computing a calibration loss Ladv as the Euclidian norm, L2 norm, of an expected calibration error ECE, which takes a weighted average over the perturbed outputs grouped in a predefined number M of equally spaced bins each having an associated average confidence and accuracy, where M>1;
checking whether the re-training converged to a predefined lower limit for a convergence rate;
first time updating weights of the AI model based on the combined loss L and a predetermined training rate η, where 0<η<=1, in case the training did not converge;
second time updating the weights of the AI model based on the calibration loss Ladv weighted with a predetermined second loss factor λadv, where 0<=λadv<=1, and the predetermined training rate η, in case the training did not converge; and
stopping the training of the AI model in case the training converged.

### Claim 7 (depends on 3)

The method according to claim 3, wherein the artificial intelligence model is a neural network.

### Claim 8 (depends on 1)

The method according to claim 1, wherein transforming comprises post-processing an output of the AI model.

### Claim 9 (depends on 8)

The method according to claim 8, wherein during the step of post-processing, parameters of a monotonic function used to transform unnormalized logits are determined by optimizing a calibration metric based on the generic samples.

### Claim 10 (depends on 8)

The method according to claim 8, wherein the artificial intelligence model is a classifier.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the validation data set is a sub-set of the training data of the trained artificial intelligence model.

### Claim 12 (depends on 1)

The method according to claim 1, wherein the validation data set is generated by modifying the training data of the trained artificial intelligence model.

### Claim 13 (depends on 1)

The method according to claim 1, wherein the transformed artificial intelligence model is provided via the user interface of the webservice platform.

### Claim 14 (depends on 1)

A computer program product, comprising a computer readable hardware storage device having computer readable program code stored therein, said program code executable by a processor of a computer system to implement the method according to claim 1.

### Claim 15

A system for transforming a trained artificial intelligence model into a trustworthy artificial intelligence model, comprising:
a user interface component to enable provision of the trained artificial intelligence model,
a memory storing the trained artificial intelligence model and user assignment information, and
a computing component for generating generic samples based on a validation data set, wherein the validation data set is determined based on training data of the trained artificial intelligence model, wherein for generating the generic samples the validation data set is modified by a domain-drift, and for transforming the trained artificial intelligence model by optimizing a calibration based on the generic samples, wherein by optimizing the calibration, an uncertainty-awareness is represented in a confidence-level for any of the generic samples.

### Claim 16 (depends on 15)

The system according to claim 15, wherein the user interface component is accessible via a webservice.

### Claim 17 (depends on 15)

The system according to claim 15, wherein the memory and the computing component are implemented on a cloud platform.

### Claim 18

A computer-implemented method for transforming a trained artificial intelligence model into a trustworthy artificial intelligence model, comprising:
providing the trained artificial intelligence model via a user interface of a webservice platform,
providing a validation data set, which is based on training data of the trained artificial intelligence model,
generating generic samples by a computing component of the webservice platform based on the validation data set,
transforming the trained artificial intelligence model by optimizing a calibration based on the generic samples, wherein transforming comprises performing a re-training of the AI-model with applying an entropy-based loss term which encourages uncertainty-awareness,
generating current outputs of the AI model for the validation data set by forward propagating validation input data of the validation data set in the AI model;
computing a categorical cross-entropy loss LCCE for the validation data set based on the current outputs and corresponding ground truth data of the validation data set;
computing a predictive entropy loss LS by removing non-misleading evidence from the current outputs and distributing the remaining current outputs over a predetermined number C of classes;
computing a combined loss L by adding to the categorical cross-entropy loss LCCE the predictive entropy loss LS weighted with a predetermined first loss factor λS, where 0<=λS<=1;
checking whether the re-training converged to a predefined lower limit for a convergence rate;
updating weights of the AI model based on the combined loss L and a predetermined training rate η, where 0<η<=1, in case the re-training did not converge; and
stopping the re-training of the AI model in case the re-training converged.

### Claim 19 (depends on 18)

The method according to claim 18, further comprising the steps of:
generating perturbed outputs of the AI model for the generic samples by forward propagating the generic input data Xadv of the generic samples in the AI model;
computing a calibration loss Ladv as the Euclidian norm, L2 norm, of an expected calibration error ECE, which takes a weighted average over the perturbed outputs grouped in a predefined number M of equally spaced bins each having an associated average confidence and accuracy, where M>1;
checking whether the re-training converged to a predefined lower limit for a convergence rate;
first time updating weights of the AI model based on the combined loss L and a predetermined training rate η, where 0<η<=1, in case the training did not converge;
second time updating the weights of the AI model based on the calibration loss Ladv weighted with a predetermined second loss factor λadv, where 0<=λadv<=1, and the predetermined training rate η, in case the training did not converge; and
stopping the training of the AI model in case the training converged.
