# US 2021/0232967 A1 · Methods and systems for tuning a static model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0232967 A1
- Document ID: US20210232967A1
- Status: Published application
- Kind: A1
- Application number: 16/775,180
- Filed: 2020-01-28
- Published: 2021-07-29
- Earliest claimed: 2020-01-28
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Katelyn Nye; Gopal Avinash; Pal Tegzes; Gireesha Rao
- Web page: https://patentgrep.com/patent/US20210232967A1

## CPC

- G16H 30/40: for processing medical images, e.g. editing
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/285: {Selection of pattern recognition techniques, e.g. of classifiers in a multi-classifier system}
- G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
- G06N 5/02: Knowledge representation; Symbolic representation
- G06N 20/00: Machine learning
- G06T 7/0012: {Biomedical image inspection}
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/50: for simulation or modelling of medical disorders
- G06T 2207/10116: X-ray image
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Methods and systems are provided for tuning a static model with multiple operating points to adjust model performance without retraining the model or triggering a new regulatory clearance. In one embodiment, a method comprises, responsive to a request to tune a model, obtaining a tuning dataset including a set of medical images, executing the model using the set of medical images as input to generate model tuning output, and determining, for each operating point of a set of operating points, a set of tuning metric values based on the tuning dataset and the model tuning output relative to each operating point. An operating point from the set of operating points may be selected based on each set of tuning metric values and, upon a request to analyze a subsequent medical image, a representation of a finding output from the static model executed at the selected operating point.

## Claims

### Claim 1

A method, comprising:
responsive to a request to tune a static model, obtaining a tuning dataset including a set of medical images;
executing the static model using the set of medical images as input to generate model tuning output;
determining, for each operating point of a set of operating points, a set of tuning metric values based on the tuning dataset and the model tuning output relative to each operating point;
selecting an operating point from the set of operating points based on each set of tuning metric values; and
upon a request to analyze a subsequent medical image, outputting a representation of a finding output of the static model executed at the selected operating point.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining the tuning dataset including the set of medical images comprises obtaining the set of medical images and a corresponding set of labels, each medical image of the set of medical images associated with a respective label.

### Claim 3 (depends on 2)

The method of claim 2, wherein each label indicates a presence or absence of a specific finding and/or a region of interest in the associated medical image.

### Claim 4 (depends on 3)

The method of claim 3, wherein the set of operating points includes a set of thresholds, each set of tuning metric values determined by comparing the model tuning output to a different threshold of the set of thresholds.

### Claim 5 (depends on 4)

The method of claim 4, wherein determining the set of tuning metric values for each operating point based on the tuning dataset and the model tuning output relative to each operating point comprises, for a first operating point of the set of operating points, determining a respective first tuning metric value for each medical image based on whether an indication of a presence or absence of a specific finding determined based on the model tuning output for that medical image relative to the first operating point matches an indication of a presence or absence of the specific finding of the label corresponding to that medical image.

### Claim 6 (depends on 5)

The method of claim 5, wherein selecting the operating point from the set of operating points based on the set of tuning metric values comprises, for each set of tuning metric values, summing the tuning metric values in that set to generate a summary score for each operating point, and selecting the operating point having a summary score that is the closest to a target score.

### Claim 7 (depends on 6)

The method of claim 6, wherein outputting the representation of the finding output of the static model executed at the selected operating point comprises outputting a notification for display on a display device indicating a presence or absence of the specific finding in the subsequent medical image.

### Claim 8 (depends on 1)

The method of claim 1, wherein outputting the representation of the finding output of the static model executed at the selected operating point comprises outputting a notification for display on a display device indicating whether or not a patient imaged in the subsequent medical image is positioned at a target position.

### Claim 9

A method, comprising:
tuning an operating point of a static model configured to output a presence or absence of a specific finding in one or more medical images by executing the static model on a set of annotated medical images and comparing output from the static model for each annotated medical image of the set of annotated medical images to each of a plurality of possible operating points, and selecting the operating point from the plurality of possible operating points that results in a target tuning metric; and
executing the static model on a subsequent medical image to determine a presence or absence of the specific finding in the subsequent medical image by comparing output of the static model for the subsequent medical image to the selected operating point.

### Claim 10 (depends on 9)

The method of claim 9, wherein the target tuning metric comprises a target sensitivity, a target specificity, a target accuracy, a target positive predictive value, and/or a target negative predictive value.

### Claim 11 (depends on 9)

The method of claim 9, wherein the target tuning metric comprises maximum accuracy and wherein selecting the operating point from the plurality of possible operating points that results in maximum accuracy comprises:
comparing the output from the static model for each annotated medical image of the set of annotated medical images to a first possible operating point to determine, for each annotated medical image, whether that annotated medical image is positive or negative for the specific finding;
assigning a first tuning metric value to each annotated medical image based on whether the determination of the positive or negative for the specific finding for each annotated medical image matches an indication of whether that annotated medical image is positive or negative for the specific finding as conveyed by an annotation of that annotated medical image;
summing each tuning metric value to determine a summary score for the first operating point;
determining a summary score for each additional possible operating point by comparing the output from the static model for each annotated medical image to each additional possible operating point and assigning a respective second tuning metric value to each annotated medical image for each additional possible operating point; and
selecting the possible operating point that has the lowest summary score.

### Claim 12 (depends on 9)

The method of claim 9, wherein selecting the operating point from the plurality of possible operating points that results in the target tuning metric comprises: determining a specificity value and a sensitivity value for each possible operating point based on the output from the static model for each annotated medical image relative to each of a plurality of possible operating points and further based on, for each annotated medical image, whether that annotated medical image is positive or negative for the specific finding as conveyed by an annotation of that annotated medical image;
plotting each specificity value as a function of a corresponding sensitivity value to form a metric curve;
outputting the metric curve for display on a display device;
receiving a user input selecting a point on the metric curve; and
setting the selected operating point as the operating point corresponding to the selected point on the metric curve.

### Claim 13 (depends on 9)

The method of claim 9, wherein the static model is configured to output a presence or absence of a specific diagnostic finding in one or more x-ray images, wherein the set of annotated medical images comprises a set of annotated x-ray images, each x-ray image of the set of annotated x-ray image including an annotation from an expert indicating a presence or absence of the specific diagnostic finding in that x-ray image.

### Claim 14

A system, comprising:
a display device; and
non-transitory memory storing instructions executable by a processor to:
execute a static model using a set of medical images of a tuning dataset as input to generate model tuning output;
determine, for each operating point of a set of operating points, a set of tuning metric values based on the tuning dataset and the model tuning output relative to each operating point;
select an operating point from the set of operating points based on each set of tuning metric values; and
upon a request to analyze a subsequent medical image, display on the display device a finding of the subsequent medical image, the finding based on an output of the static model relative to the selected operating point.

### Claim 15 (depends on 14)

The system of claim 14, wherein the memory and processor are included in an x-ray imaging device, and wherein the subsequent medical image is acquired by the x-ray imaging device.

### Claim 16 (depends on 15)

The system of claim 15, wherein at least some images of the set of medical images are acquired with the x-ray imaging device.

### Claim 17 (depends on 14)

The system of claim 14, wherein the memory and the processor are included in a computing device operably coupled to an imaging device, and wherein the subsequent medical image is acquired with the imaging device.

### Claim 18 (depends on 14)

The system of claim 14, wherein selecting the operating point from the set of operating points based on each set of tuning metric values comprises selecting the operating point from the set of operating points based on each set of tuning metric values and further based on a target tuning metric.

### Claim 19 (depends on 18)

The system of claim 18, wherein the target turning metric comprises a target sensitivity, a target specificity, a target accuracy, a target positive predictive value, and/or a target negative predictive value.

### Claim 20 (depends on 14)

The system of claim 14, wherein the finding of the subsequent medical image is determined by comparing the output of the static model to the selected operating point and selecting the finding based on whether the output of the static model is greater than the selected operating point.
