# US 2023/0057653 A1 · Method and system and apparatus for quantifying uncertainty for medical image assessment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0057653 A1
- Document ID: US20230057653A1
- Status: Published application
- Kind: A1
- Application number: 17/819,437
- Filed: 2022-08-12
- Published: 2023-02-23
- Earliest claimed: 2021-08-23
- Assignees: Siemens Healthcare GmbH
- Applicants: Siemens Healthcare GmbH
- Inventors: Florin-Cristian Ghesu; Awais Mansoor; Sasa Grbic; Ramya Vunikili; Sanjeev Kumar Karn; Rajeev Bhatt Ambati; Oladimeji Farri; Bogdan Georgescu; Dorin Comaniciu
- Web page: https://patentgrep.com/patent/US20230057653A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 30/40: for processing medical images, e.g. editing
- G16H 70/60: relating to pathologies
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and methods for providing a means for improving the expressiveness and/or robustness of a machine learning system's result, based on imaging data and/or to make it possible to combine imaging data with non-imaging data to improve statements, which are deduced from the imaging data. The object is achieved by a computer implemented method, and uncertainty quantifier, medical system and a computer program product, and includes receiving a set of input data quantified as uncertainty, providing an information fusion algorithm, and applying the received set of input data on the provided information fusion algorithm, while modeling the propagation of uncertainty through the information fusion algorithm to predict an uncertainty for the medical assessment as a result (r), provided by the machine-learning system (M), based on the provided set of input data.

## Claims

### Claim 1

A computer-implemented method for providing an uncertainty prediction for a medical assessment on imaging data being provided by a machine-learning system, the method comprising:
receiving a set of input data, comprising the imaging data, which have been provided to the machine-learning system and non-imaging data, each represented as a signal with some degree of noise, being quantified as aleatoric uncertainty, wherein the non-imaging data comprise medical or healthcare data in a digital format or representation, which do not comprise image data acquired from an imaging modality;
providing an information fusion algorithm, wherein the information fusion algorithm is an algorithm for combining different data sets, provided in different formats, including the imaging data and the non-imaging data; and
applying the received set of input data on the provided information fusion algorithm, while modeling the propagation of uncertainty through the information fusion algorithm to predict an uncertainty for the medical assessment as a result, provided by the machine-learning system, based on the provided set of input data.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the information fusion algorithm uses at least one of an information fusion model and graph neural network, being optimized for maximizing entropy in the non-imaging data.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein at least one of the entropy of the information fusion model and the graph neural network is optimized by a greedy algorithm.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the method further comprises:
applying a selection algorithm for selecting a subset of provided input data, which minimizes a cost function and/or reduces uncertainty by using a reinforcement learning model.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein input data of a set of input data sources may be present or absent and wherein the method provides a guided decision which of the absent input data sources would at least one of reduce uncertainty or minimize a cost function.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein providing input data of the set of input data sources comprises measuring or acquiring data from at least one of imaging modalities and medical databases.

### Claim 7 (depends on 4)

The computer-implemented method of claim 4, wherein a reinforcement learning model is based on a decision process
M=(S, A, T, R, η),
where S denotes a state space, A an action space, T a stochastic transition process, R a reward function and η a discount factor, wherein actions represent providing additional input data sources.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the reward function is defined to at least one of minimize the cost or the predicted uncertainty.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the non-imaging data comprises at least one of: biomarkers, clinical notes, image annotations, medical report dictations, measurements, laboratory values, diagnostic codes, data from an EHR-database (DB), and anamnestic data of a patient.

### Claim 10 (depends on 4)

The computer-implemented method of claim 4, wherein an uncertainty propagation model comprising at least one of a Bayesian deep model, Q-Learning, and actor critic learning, is used for the reinforcement learning model.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein an uncertainty propagation model is used in the information fusion model.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, wherein the information fusion model is capable of processing a situation, where a subset of input data sources is not available or only available by certain costs.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, wherein the predicted uncertainty is at least one of patient-specific, imaging data specific, and signal specific.

### Claim 14 (depends on 7)

The computer-implemented method of claim 7, wherein on a user interface, a set of interaction buttons is provided so that a user can indicate that an input data source is not available during inference or that the action space of the non-Markovian decision process is limited to the input data sources, being available so that the user may select a type of optimization and in particular if he or she wants to minimize prediction uncertainty or costs.

### Claim 15

An uncertainty quantifier for a medical assessment on imaging data being provided by a machine-learning system, the uncertainty quantifier comprising:
an input interface for connecting to a set of input data sources for receiving a set of input data, comprising the imaging data, which have been provided to the machine-learning system and non-imaging data, each represented as a signal with noise, being quantified as uncertainty, in particular aleatoric uncertainty, wherein the non-imaging data comprise medical or healthcare data in a digital format or representation, which do not comprise image data acquired from an imaging modality;
a storage for storing an information fusion algorithm, wherein the information fusion algorithm is an algorithm for combining different data sets, provided in different formats, including the imaging data and the non-imaging data;
a processing unit which is configured for applying the received set of input data on the provided information fusion algorithm while modeling the propagation of uncertainty through the information fusion algorithm to predict uncertainty of the medical assessment, which has been provided by the machine-learning system, based on the provided set of input data; and
an output interface for providing the predicted uncertainty as result.

### Claim 16 (depends on 15)

A medical system for a medical assessment on imaging data being provided by a machine-learning system with a set of medical input data sources and with an uncertainty quantifier according to claim 15.

### Claim 17 (depends on 15)

The uncertainty quantifier of claim 15, wherein the processing unit is further configured for applying a selection algorithm for selecting a subset of provided input data, which at least one of: minimizes a cost function and reduces uncertainty by using a reinforcement learning model.

### Claim 18 (depends on 15)

The uncertainty quantifier of claim 15, wherein a reinforcement learning model is based on a decision process
M=(S, A, T, R, η),
where S denotes a state space, A an action space, T a stochastic transition process, R a reward function and η a discount factor, wherein actions represent providing additional input data sources.

### Claim 19

A non-transitory computer readable medium storing computer program instructions, the computer program instructions when executed by a processor cause the processor to perform operations comprising:
receiving a set of input data, comprising imaging data, which have been provided to a machine-learning system and non-imaging data, each represented as a signal with some degree of noise, being quantified as uncertainty, in particular aleatoric uncertainty, wherein the non-imaging data comprise medical or healthcare data in a digital format or representation, which do not comprise image data acquired from an imaging modality;
providing an information fusion algorithm, wherein the information fusion algorithm is an algorithm for combining different data sets, provided in different formats, including the imaging data and the non-imaging data; and
applying the received set of input data on the provided information fusion algorithm, while modeling the propagation of uncertainty through the information fusion algorithm to predict an uncertainty for the medical assessment as a result, provided by the machine-learning system, based on the provided set of input data.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, wherein the operations further comprise applying a selection algorithm for selecting a subset of provided input data, which at least one of: minimizes a cost function and reduces uncertainty by using a reinforcement learning model.
