# US 2021/0174485 A1 · Systems and methods for detecting anomalies using image based modeling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174485 A1
- Document ID: US20210174485A1
- Status: Published application
- Kind: A1
- Application number: 16/707,585
- Filed: 2019-12-09
- Published: 2021-06-10
- Earliest claimed: 2019-12-09
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Sridhar Nuthi
- Web page: https://patentgrep.com/patent/US20210174485A1

## CPC

- G16H 40/40: for the management of medical equipment or devices, e.g. scheduling maintenance or upgrades
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/2433: Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
- G06T 7/0002: {Inspection of images, e.g. flaw detection}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/7784: {based on feedback from supervisors}
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G06T 2207/20081: Training; Learning
- G06T 2207/30242: Counting objects in image
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for detecting anomalies in a system. The method includes collecting training data from the system, converting the training data into training images using an image generator, and designating each of the training images as corresponding to events for the system, where the events are at least one of an expected normal event and a non-normal event. The method further includes generating an image recognition model based on the training images and the designations thereof. The method further includes collecting new data from the system, converting the new data into input images, and analyzing the input images using the image recognition model to determine which of the events for the system are represented in the input images, where the anomalies are detected when the input images are determined to at least one of represent a non-normal event and fail to represent an expected normal event.

## Claims

### Claim 1

A method for detecting anomalies in a system, the method comprising:
collecting training data from the system;
converting the training data into training images using an image generator;
designating each of the training images as corresponding to events for the system, wherein the events are at least one of an expected normal event and a non-normal event;
generating an image recognition model based on the training images and the designations thereof;
collecting new data from the system;
converting the new data into input images; and
analyzing the input images using the image recognition model to determine which of the events for the system are represented in the input images;
wherein the anomalies are detected when the input images are determined to at least one of represent a non-normal event and fail to represent an expected normal event.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the training data and the new data include system logs generated during operation of the system.

### Claim 3 (depends on 1)

The method according to claim 1, further comprising identifying a pattern in at least one of the training data and the new data and modifying an at least one of the training images and the input images corresponding thereto to demarcate the identification of the pattern therein.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the modification is color coding.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the pattern corresponds to the system starting up.

### Claim 6 (depends on 3)

The method according to claim 3, further comprising counting a number of times the pattern is identified over a period of time.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising identifying a pattern in at least one of the training data and the new data, and further comprising removing the pattern from at least one of the training data and the new data prior to converting into the training images and the input images, respectively.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the image recognition model is generated via machine learning.

### Claim 9 (depends on 8)

The method according to claim 8, further comprising updating the image recognition model by adding a portion of the new data to the training data.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the machine learning process is supervised such that user feedback is used to designate the events corresponding to the portion of the new data being added to the training data.

### Claim 11 (depends on 1)

The method according to claim 1, further comprising identifying probable root causes for the anomalies detected.

### Claim 12 (depends on 11)

The method according to claim 11, further comprising determining when system changes are made, and further comprising determining whether the system changes resolve the anomalies detected.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the probable root causes are stored in a root case database, further comprising updating the probably root causes database based on the determination of whether the system changes resolved the anomalies detected.

### Claim 14 (depends on 1)

The method according to claim 1, further comprising trending changes in at least one of the training images and the input images over time.

### Claim 15 (depends on 1)

The method according to claim 1, wherein each of the input images represents the new data generated by the system over one day, and wherein multiple of the input images are combined analyzing together using the image recognition model.

### Claim 16 (depends on 1)

The method according to claim 1, wherein the new data is in text form.

### Claim 17 (depends on 1)

The method according to claim 1, wherein the system is a medical imaging system that produces imaging data, and wherein the training data and the new data include the imaging data produced by the medical imaging system.

### Claim 18 (depends on 1)

The method according to claim 1, further comprising identifying an experiment expected to impact the new data produced by the system for testing the image recognition model, and further comprising changing the system according to the experiment and updating the image recognition model according to the impact thereof.

### Claim 19

A system for detecting anomalies in a medical imaging system, the system comprising:
an image recognition model based on training images converted from training data produced by the system, wherein the training data represents events occurring in the system, and wherein the events include at least one of expected normal events and non-normal events;
a data aggregator that collects and aggregates new data from the system, wherein the new data also reflect the events occurring in the system;
an image generator that converts the new data into input images;
a computing system that analyzes the input images using the image recognition model to determine which of the events for the system are represented in the input images;
wherein the anomalies are detected when the input images are determined to at least one of represents one of the non-normal events and fails to represent one of the expected normal events.

### Claim 20

A method for detecting anomalies in a medical imaging system, the method comprising:
collecting training data from the medical imaging system;
converting the training data into training images using an image generator;
designating each of the training images as corresponding to events for the medical imaging system, wherein the events are at least one of an expected normal event and a non-normal event;
generating an image recognition model via machine learning based on the training images and the designations thereof;
collecting new data from the medical imaging system, wherein the new data includes environmental data for the medical imaging system and logs for applications running on the medical imaging system;
converting the new data into input images;
identifying a pattern in the new data and modifying the input images corresponding thereto and demarcating the identification of the pattern therein; and
analyzing the input images using the image recognition model to determine which of the events for the system are represented in the input images;
wherein the anomalies are detected when the input images are determined to at least one of represent a non-normal event and fail to represent an expected normal event.
