# US 2023/0274818 A1 · Image analysis and insight generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274818 A1
- Document ID: US20230274818A1
- Status: Published application
- Kind: A1
- Application number: 18/297,051
- Filed: 2023-04-07
- Published: 2023-08-31
- Earliest claimed: 2022-01-25
- Assignees: Northwestern Memorial Healthcare
- Applicants: Northwestern Memorial Healthcare
- Inventors: Mozziyar ETEMADI
- Web page: https://patentgrep.com/patent/US20230274818A1

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G06T 7/0012: {Biomedical image inspection}
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Approaches for analyzing an input image and providing one or more outputs related to the input image are provided. In accordance with an exemplary embodiment, an input image may be received and analyzed, using a trained machine learning model, to generate an inference related to the image. Based, at least in part, upon the generated inference, one or more reports related to the inference can be generated and provided for presentation on a user device. A user can interact with the report in a conversational manner with the computer system to generate additional reports or insights related to the input image.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving an input image;
receiving a request for at least one insight related to the input image;
analyzing the input image to detect one or more anomalies present in the input image;
utilizing at least one neural network to infer information associated with the input image;
generating a plurality of reports related to the input image based, at least in part, upon the inferred information, wherein each of the plurality of reports have at least one associated metric.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the at least one neural network is trained using at least one of an image model, a text model, and historical data.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising:
enabling a user to interact with at least a portion of one of a report in the plurality of reports and the input image; and
providing an indication on an other of the report and the input image corresponding to the interacted portion of the report.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, further comprising:
enabling a user to interact with the plurality of reports, wherein an interaction includes one or more of: highlighting in the input image or a report of the plurality of reports by a user, receiving a question associated with the plurality of reports from a user, or an interaction with a distribution generated for the plurality of reports.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the input image is obtained through a medical imaging process.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of reports are provided in a histogram, wherein the histogram is configured to provide a comprehensive presentation of the plurality of reports.

### Claim 7

A computer-implemented method, comprising:
receiving an input image;
generating, using a machine learning model, at least one inference associated with the image;
providing a first output for display based, at least in part, upon the at least one inference;
receiving at least one interaction with the first output; and
providing at least one second output for display in response to the received at least one interaction.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the first output includes a set of reports associated with the at least one inference.

### Claim 9 (depends on 7)

The computer-implemented method of claim 7, wherein the at least one second output includes one or more of: an answer in response to the at least one interaction, and a new set of reports generated in response to the at least one interaction.

### Claim 10 (depends on 7)

The computer-implemented method of claim 7, wherein the at least one interaction includes one or more of: highlighting in the first output by a user, receiving a question from a user, or an interaction with a distribution generated for the first output.

### Claim 11 (depends on 7)

The computer-implemented method of claim 7, further comprising:
enabling a user to interact with at least a portion of a report in the first output; and
providing an indication on the input image corresponding to the interacted portion of the report.

### Claim 12 (depends on 7)

The computer-implemented method of claim 7, wherein the input image is obtained through a medical imaging process.

### Claim 13 (depends on 7)

The computer-implemented method of claim 7, wherein the machine learning model is trained using at least one of an image model, a text model, and historical data.

### Claim 14 (depends on 7)

The computer-implemented method of claim 7, wherein the first output is provided as a histogram, wherein the histogram is configured to provide a comprehensive presentation of a plurality of reports.

### Claim 15 (depends on 10)

The computer-implemented method of claim 10, wherein the distribution generated for the first output provides insight as to a clinical diagnosis associated with the input image.

### Claim 16

A computer-implemented method, comprising:
training a machine learning model on a set of training data, wherein the set of training data includes individual token values and one or more groupings of the individual token values;
receiving an input image;
analyzing, using the trained machine learning model, the input image;
receiving an output associated with the analyzed input image;
sampling at least a first subset of data received in the output, wherein a second subset of data in the output is excluded from the sampling; and
generating a report associated with the input image based, at least in part, upon the sampled first subset of data.

### Claim 17 (depends on 16)

The computer-implemented method of claim 16, wherein the individual token values correspond to text values or image values in the set of training data.

### Claim 18 (depends on 16)

The computer-implemented method of claim 16, wherein the output corresponds to a distribution of text values.

### Claim 19 (depends on 16)

The computer-implemented method of claim 16, wherein the output associated with the analyzed image is overfit to the set of training data.

### Claim 20 (depends on 19)

The computer-implemented method of claim 19, wherein overfitting the output to one or more modalities in the set of training data increases a level of accuracy associated with the report.
