# US 2019/0171914 A1 · Systems and user interfaces for enhancement of data utilized in machine-learning based medical image review

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0171914 A1
- Document ID: US20190171914A1
- Status: Published application
- Kind: A1
- Application number: 15/831,057
- Filed: 2017-12-04
- Published: 2019-06-06
- Earliest claimed: 2017-12-04
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Aviad Zlotnick; Alon Hazan; Murray A. Reicher
- Web page: https://patentgrep.com/patent/US20190171914A1

## CPC

- G06V 30/19167: {Active pattern learning}
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F 18/2178: {based on feedback of a supervisor}
- G06F 18/24: Classification techniques
- G06F 18/41: {Interactive pattern learning with a human teacher}
- G06T 7/0014: {using an image reference approach}
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 30/19133: {Interactive pattern learning with a human teacher}
- G06V 30/19173: {Classification techniques}
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G16H 40/63: for local operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and techniques are disclosed for improvement of machine learning systems based on enhanced training data. An example method includes generating an interactive classification user interface concurrently displaying a first group of medical images and a second group of medical images, each group depicting objects associated with a respective classification. User input indicating movement of medical images from the first group to the second group is detected. The moved medical images are classified according to the second group. The re-classified medical images are provided to a machine learning system, with the machine learning system updating based on analysis of object characteristics of the re-classified medical images to increase accuracies associated with automated assignment of classifications.

## Claims

### Claim 1

A method for facilitating proper classification of features in medical images comprising:
generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying a first group of medical images and a second group of medical images, wherein each of the first group of medical images depicts an object associated with a first classification and each of the second group of medical images depicts an object associated with a second classification;
detecting user input indicating movement of one or more medical images from the first group to the second group;
in response to the user input, re-classifying the one or more medical images as associated with the second classification; and
providing the re-classifications to a machine learning system.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first group of medical images is displayed as a grid.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first group of medical images is displayed as a stack.

### Claim 4 (depends on 1)

The method of claim 1, wherein each medical image is associated with a user who classified the medical image, and wherein the method further comprises:
accessing information indicating users associated with the moved medical images; and
generating respective notifications for presentation to the indicated users identifying the re-classification.

### Claim 5 (depends on 1)

The method of claim 1, wherein, the machine learning system updates based on analysis of object characteristics of the re-classified one or more medical images to increase an accuracy associated with automated assignment of classifications.

### Claim 6

A method for facilitating proper classification of features in medical images comprising:
generating, for display on a display of a computing system, an interactive classification user interface concurrently displaying two groups of medical images, each group having been assigned a disparate classification relating to objects included in the medical images, and each object having been extracted from a larger medical image;
the interactive classification user interface enabling a user to adjust a classification of each medical image to correspond to a different group, such that the medical image is re-classified according to the different classification; and
providing the re-classifications to a machine learning system, the machine learning system updating based on the re-classifications to increase an accuracy associated with assigning classifications.

### Claim 7 (depends on 6)

The method of claim 6, wherein borders of the objects included in the medical images are highlighted, the highlighting being automatically generated by the machine learning system.

### Claim 8 (depends on 6)

The method of claim 6, wherein the interactive classification user interface displays each group of medical images as a respective grid.

### Claim 9 (depends on 6)

The method of claim 6, wherein the interactive classification user interface displays each group of medical images as a respective stack.

### Claim 10 (depends on 6)

The method of claim 6, wherein the interactive classification user interface receives input from the user on a touch-screen display, the input representing the user dragging a first image included in a first group to a second group, such that the first image is re-classified.

### Claim 11 (depends on 6)

The method of claim 6, wherein the interactive classification user interface is configured to receive input via a touch-sensitive display, and wherein received input directed to a particular medical image causes the interactive classification user interface to update to present a full image context related to the particular medical image.

### Claim 12 (depends on 6)

The method of claim 6, wherein a re-classification assigned by the user is stored, and wherein an initial user who classified the associated medical image is alerted.

### Claim 13 (depends on 6)

The method of claim 6, wherein a threshold number of users assign classifications to the medical images, and wherein a final classification is determined for each medical image prior to providing the classifications to the machine learning system.

### Claim 14 (depends on 6)

The method of claim 6, wherein the classifications relate to a diagnosis associated with an object, a shape of an object, or a change in size or character of an object over time.

### Claim 15

Non-transitory computer storage-media storing instructions that when executed by a system of one or more computers, cause the one or more computers to perform operations comprising:
providing a visual concurrent display of two groups of medical images, each group being associated with a respective classification, wherein the classification relates to objects included in the medical images;
providing a user interface enabling a reviewing user to re-classify medical images, such that one or more medical images are assigned to a different group;
responsive to the reviewing user utilizing the user interface to indicate that a particular medical image is to be re-classified, updating the particular image to be assigned to the different group and causing storage of the update.

### Claim 16 (depends on 15)

The computer storage-media of claim 15, wherein the operations further comprise:
training a machine learning system based on the re-classification.

### Claim 17 (depends on 15)

The computer storage-media of claim 15, wherein each group is presented as a respective grid of medical images.

### Claim 18 (depends on 15)

The computer storage-media of claim 15, wherein each group is presented as a respective stack of medical images.

### Claim 19 (depends on 15)

The computer storage-media of claim 15, wherein an initial reviewing user who classified the particular medical image is alerted.

### Claim 20 (depends on 15)

The computer storage-media of claim 15, wherein a notification is provided to a third reviewing user, the notification enabling activation of an application on a user device of the third reviewing user to determine a final classification for the particular medical image.
