# US 2019/0172581 A1 · Systems and user interfaces for enhancement of data utilized in machine-learning based medical image review

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0172581 A1
- Document ID: US20190172581A1
- Status: Published application
- Kind: A1
- Application number: 15/831,117
- Filed: 2017-12-04
- Published: 2019-06-06
- Earliest claimed: 2017-12-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Aviad Zlotnick; Ayelet Akselrod-Ballin; Murray A. Reicher; Sivan Ravid
- Web page: https://patentgrep.com/patent/US20190172581A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06N 20/10: using kernel methods, e.g. support vector machines [SVM]
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/30: for calculating health indices; for individual health risk assessment
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Systems and techniques are disclosed for improvement of machine learning systems based on enhanced training data. An example method includes accessing a database storing associations between objects included in medical images and classifications of the objects. A risk assessment model adapted to determine a risk condition for an object is accessed, the assessment based on features of the object. Risk conditions associated with respective objects are determined based on the risk assessment model. A group of objects associated with a first risk condition is identified. An interactive user interface is generated for display, the user interface concurrently displaying images of the group of objects. The interactive user interface enables a user to select subsets of images to be concurrently assigned a user-selected classification. User selected classifications are provided to a machine learning system adapted to update the risk assessment model based on the classifications to increase accuracy of the model.

## Claims

### Claim 1

A method of facilitating proper classification of features in medical images comprising:
accessing an object classification database storing associations between objects included in medical images and classifications of the objects;
accessing a risk assessment model adapted to determine a risk condition for an object based on one or more of automatic analysis of features of the object and patient history;
determining, based on the risk assessment model, risk conditions associated with respective objects;
identifying a group of the objects each associated with a first risk condition;
generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying images of the group of objects;
the interactive classification user interface enabling a user to select subsets of the images to be concurrently assigned a user-selected classification; and
providing the user selected classifications to a machine learning system.

### Claim 2 (depends on 1)

The method of claim 1, wherein the images of the group of objects are displayed as a grid.

### Claim 3 (depends on 1)

The method of claim 1, wherein the images of the group of objects are displayed as a stack.

### Claim 4 (depends on 1)

The method of claim 1, wherein the interactive classification user interface:
displays user interface elements on which an image can be dragged, each user interface element being associated with a particular classification; and
in response to an image being dragged onto a particular user interface element, assigning to the image to be classified according to the particular classification.

### Claim 5 (depends on 1)

The method of claim 1, wherein the interactive classification user interface is configured to receive input via a touch-sensitive display, and wherein received input directed to a particular image causes the interactive classification user interface to update to present a full image context related to the particular image.

### Claim 6 (depends on 1)

The method of claim 1, wherein the machine learning system is adapted to update the risk assessment model based on the classifications to increase accuracy of risk conditions determined by the risk assessment model.

### Claim 7

A method for facilitating proper classification of features in medical images comprising:
generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying each of a group of medical images associated with patients, the group including medical images assigned a same risk associated with a medical condition and the risk being determined based, at least in part, on machine-learning based analysis of medical records of the patients and/or analysis of the medial images, the medical images including objects to be classified;
the interactive classification user interface enabling the user to select subsets of the images to be concurrently assigned classifications, each classification indicating a respective diagnosis associated with the medical condition; and
providing the classifications of the objects to a machine learning system, the machine learning system updating based on the classifications to increase an accuracy associated with determining risk.

### Claim 8 (depends on 7)

The method of claim 7, wherein borders of the objects included in the medical images are highlighted, the highlighting being automatically generated by the machine learning system.

### Claim 9 (depends on 7)

The method of claim 7, wherein the interactive classification user interface displays the medical images as a grid.

### Claim 10 (depends on 7)

The method of claim 7, wherein the interactive classification user interface displays the medical images as a stack.

### Claim 11 (depends on 10)

The method of claim 10, wherein the medical images displayed as a stack cycle periodically, such that a top presented medical image on the stack is replaced with a subsequent medical image after a threshold amount of time.

### Claim 12 (depends on 7)

The method of claim 7, wherein the interactive classification user interface is configured to receive input via a touch-sensitive display, and wherein received input directed to a particular medical image causes the interactive classification user interface to update to present a full image context related to the particular medical image.

### Claim 13 (depends on 12)

The method of claim 12, wherein the full image context includes one or more of information associated with a patient, risk information associated with the medical condition that is related to the patient, or information describing risk associated with the particular medical image.

### Claim 14 (depends on 7)

The method of claim 7, wherein risk is a BIRADS score and the diagnosis is an object being malignant or benign.

### Claim 15 (depends on 7)

The method of claim 7, wherein a threshold number of users assign classifications to the medical images, and wherein a final classification is determined for each medical image prior to providing the classifications to the machine learning system.

### Claim 16 (depends on 15)

The method of claim 15, wherein the determination of a final classification is based at least in part on a performance associated with each user, the performance being based on a performance in assigning a correct diagnosis to control images of known classifications, the control images being included in the group.

### Claim 17

Non-transitory computer storage-media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:
obtaining medical records for a plurality of patients and one or more medical images associated with the patients, the medical images including objects to be classified;
determining, via a machine learning system based on the obtained information, risks associated with the objects, the risks being associated with one or more medical conditions;
generating, for each medical condition, groups of medical images, each group including medical images with a same associated risk; and
presenting, via an interactive user interface, medical images included in a same group, the interactive user interface enabling a classification of the objects, wherein the classification indicates a particular diagnosis for a medical condition.

### Claim 18 (depends on 17)

The computer storage-media of claim 17, wherein the operations further comprise:
receiving classifications of one or more objects contrary to a determined risk associated with the medical images, and training the machine learning system based on the received classifications.

### Claim 19 (depends on 17)

The computer storage-media of claim 17, wherein the medical images included in the same group are presented as a grid.

### Claim 20 (depends on 17)

The computer storage-media of claim 17, wherein the medical images included in the same group are presented as a stack.
