# US 2025/0149169 A1 · Learnable visual prompt engineering

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0149169 A1
- Document ID: US20250149169A1
- Status: Published application
- Kind: A1
- Application number: 18/504,649
- Filed: 2023-11-08
- Published: 2025-05-08
- Earliest claimed: 2023-11-08
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Deepa Anand; Dattesh Shanbhag; Hariharan Ravishankar; Suresh Emmanuel Devadoss Joel; Rakesh Mullick; Rachana Sathish; Rahul Venkataramani; Krishna Seetharam Shriram; Prasad Sudhakara Murthy
- Web page: https://patentgrep.com/patent/US20250149169A1

## CPC

- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T 7/0012: {Biomedical image inspection}
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30004: Biomedical image processing
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems or techniques for facilitating learnable visual prompt engineering are provided. In various embodiments, a system can access a medical image and a pre-trained machine learning model that is configured to perform a diagnostic or prognostic inferencing task. In various aspects, the system can apply a pre-processing transformation to one or more pixels or voxels of the medical image, thereby yielding a transformed version of the medical image, wherein the pre-processing transformation can convert an input pixel or voxel intensity value to an output pixel or voxel intensity value via one or more parameters that are iteratively learned. In various instances, the system can perform the diagnostic or prognostic inferencing task, by executing the pre-trained machine learning model on the transformed version of the medical image.

## Claims

### Claim 1

A system, comprising:
a processor that executes computer-executable components stored in a non-transitory computer-readable memory, wherein the computer-executable components comprise:
an access component that accesses a medical image and a pre-trained machine learning model that is configured to perform a diagnostic or prognostic inferencing task;
a visual prompt engineering component that applies a pre-processing transformation to one or more pixels or voxels of the medical image, thereby yielding a transformed version of the medical image, wherein the pre-processing transformation converts an input pixel or voxel intensity value to an output pixel or voxel intensity value via one or more parameters that are iteratively learned; and
an execution component that performs the diagnostic or prognostic inferencing task, by executing the pre-trained machine learning model on the transformed version of the medical image.

### Claim 2 (depends on 1)

The system of claim 1, wherein the one or more parameters of the pre-processing transformation are iteratively learned based on an annotated training dataset.

### Claim 3 (depends on 2)

The system of claim 2, wherein the annotated training dataset comprises a training medical image corresponding to a ground-truth annotation, and wherein the computer-executable components further comprise:
a training component that:
randomly initializes the one or more parameters of the pre-processing transformation;
executes the pre-processing transformation on the training medical image, thereby yielding a first output;
executes the pre-trained machine learning model on the first output, thereby yielding a second output;
computes an error between the second output and the ground-truth annotation; and
updates, via backpropagation driven by the error, the one or more parameters of the pre-processing transformation, without updating any parameters of the pre-trained machine learning model.

### Claim 4 (depends on 1)

The system of claim 1, wherein the one or more parameters of the pre-processing transformation are iteratively learned in the absence of annotated training data, based on the medical image and based on an auxiliary confidence predictor.

### Claim 5 (depends on 4)

The system of claim 4, wherein the computer-executable components further comprise:
a training component that:
randomly initializes the one or more parameters of the pre-processing transformation;
executes the pre-processing transformation on the medical image, thereby yielding a first output;
executes the pre-trained machine learning model on the first output, thereby yielding a second output;
computes, via execution of the auxiliary confidence predictor, a confidence score for the second output; and
updates, via backpropagation driven by the confidence score, the one or more parameters of the pre-processing transformation, without updating any parameters of the pre-trained machine learning model.

### Claim 6 (depends on 1)

The system of claim 1, wherein the pre-processing transformation is a linear or non-linear function or machine learning model that takes a pixel or voxel intensity value as an argument, and wherein the one or more parameters comprise coefficients of the linear or non-linear function or machine learning model.

### Claim 7 (depends on 1)

The system of claim 1, wherein the pre-processing transformation is a color map assignment, and wherein the one or more parameters comprise an intensity window and a gamma-value of the color map assignment.

### Claim 8 (depends on 1)

The system of claim 1, wherein the diagnostic or prognostic inferencing task is medical image classification, medical image segmentation, or medical image regression.

### Claim 9

A computer-implemented method, comprising:
accessing, by a device operatively coupled to a processor, a medical image and a pre-trained machine learning model that is configured to perform a diagnostic or prognostic inferencing task;
applying, by the device, a pre-processing transformation to one or more pixels or voxels of the medical image, thereby yielding a transformed version of the medical image, wherein the pre-processing transformation converts an input pixel or voxel intensity value to an output pixel or voxel intensity value via one or more parameters that are iteratively learned; and
performing, by the device, the diagnostic or prognostic inferencing task, by executing the pre-trained machine learning model on the transformed version of the medical image.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the one or more parameters of the pre-processing transformation are iteratively learned based on an annotated training dataset.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the annotated training dataset comprises a training medical image corresponding to a ground-truth annotation, and further comprising:
randomly initializing, by the device, the one or more parameters of the pre-processing transformation;
executing, by the device, the pre-processing transformation on the training medical image, thereby yielding a first output;
executing, by the device, the pre-trained machine learning model on the first output, thereby yielding a second output;
computing, by the device, an error between the second output and the ground-truth annotation; and
updating, by the device and via backpropagation driven by the error, the one or more parameters of the pre-processing transformation, without updating any parameters of the pre-trained machine learning model.

### Claim 12 (depends on 9)

The computer-implemented method of claim 9, wherein the one or more parameters of the pre-processing transformation are iteratively learned in the absence of annotated training data, based on the medical image and based on an auxiliary confidence predictor.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, further comprising:
randomly initializing, by the device, the one or more parameters of the pre-processing transformation;
executing, by the device, the pre-processing transformation on the medical image, thereby yielding a first output;
executing, by the device, the pre-trained machine learning model on the first output, thereby yielding a second output;
computing, by the device and via execution of the auxiliary confidence predictor, a confidence score for the second output; and
updating, by the device and via backpropagation driven by the confidence score, the one or more parameters of the pre-processing transformation, without updating any parameters of the pre-trained machine learning model.

### Claim 14 (depends on 9)

The computer-implemented method of claim 9, wherein the pre-processing transformation is a linear or non-linear function or machine learning model that takes a pixel or voxel intensity value as an argument, and wherein the one or more parameters comprise coefficients of the linear or non-linear function or machine learning model.

### Claim 15 (depends on 9)

The computer-implemented method of claim 9, wherein the pre-processing transformation is a color map assignment, and wherein the one or more parameters comprise an intensity window and a gamma-value of the color map assignment.

### Claim 16 (depends on 9)

The computer-implemented method of claim 9, wherein the diagnostic or prognostic inferencing task is medical image classification, medical image segmentation, or medical image regression.

### Claim 17

A computer program product for facilitating learnable visual prompt engineering, the computer program product comprising a non-transitory computer-readable memory having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:
access an image;
generate an adapted version of the image via a pixel-to-pixel or voxel-to-voxel pre-processing transformation comprising one or more parameters that are iteratively learned; and
perform a visual inferencing task, by executing a pre-trained machine learning model on the adapted version of the image.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the one or more parameters of the pixel-to-pixel or voxel-to-voxel pre-processing transformation are iteratively learned based on an annotated training dataset.

### Claim 19 (depends on 17)

The computer program product of claim 17, wherein the one or more parameters of the pixel-to-pixel or voxel-to-voxel pre-processing transformation are iteratively learned in the absence of annotated training data, based on the image and based on an auxiliary confidence predictor.

### Claim 20 (depends on 17)

The computer program product of claim 17, wherein the pixel-to-pixel or voxel-to-voxel pre-processing transformation is a linear or non-linear function, a color map assignment, or a machine learning or deep learning model.
