# US 12,602,775 B2 · Interpolation of medical images

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,775 B2
- Document ID: US12602775B2
- Status: Granted
- Kind: B2
- Application number: 17/742,463
- Filed: 2022-05-12
- Published: 2023-11-16
- Granted: 2026-04-14
- Earliest claimed: 2022-05-12
- Assignees: Globus Medical, Inc.
- Applicants: Globus Medical, Inc.
- Inventors: Sanjay M. Joshi; Arjang Noushin; Neil Crawford
- Web page: https://patentgrep.com/patent/US12602775B2

## CPC

- G06T 7/0012: {Biomedical image inspection}
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B 90/37: {Surgical systems with images on a monitor during operation}
- G06T 3/10: Selection of transformation methods according to the characteristics of the input images
- G06T 7/33: using feature-based methods
- G06T 7/55: from multiple images
- G06T 7/73: using feature-based methods
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- A61B 34/30: Surgical robots
- A61B 2017/00207: {with hand gesture control or hand gesture recognition}
- A61B 2034/2048: {using an accelerometer or inertia sensor}
- A61B 2034/2055: {Optical tracking systems}
- A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image}
- A61B 2090/372: {Details of monitor hardware}
- A61B 2090/374: {NMR or MRI}
- A61B 2090/376: {using X-rays, e.g. fluoroscopy}
- A61B 2090/3762: {using computed tomography systems [CT]}
- A61B 2090/378: {using ultrasound}
- A61B 2090/3983: {Reference marker arrangements for use with image guided surgery}
- A61B 2090/502: {Headgear, e.g. helmet, spectacles}
- A61B 2560/0437: Trolley or cart-type apparatus
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/10088: Magnetic resonance imaging [MRI]
- G06T 2207/10116: X-ray image
- G06T 2207/10124: Digitally reconstructed radiograph [DRR]
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system can be configured to generate a medical image. The system can include processing circuitry and memory coupled to the processing circuitry. The memory can include instructions stored therein that are executable by the processing circuitry to cause the system to perform operations. The operations can include obtaining a first image of an anatomical object. The operations can further include extracting first information associated with the anatomical object from the first image. The operations can further include obtaining a second image of the anatomical object. The operations can further include extracting second information associated with the anatomical object from the second image. The operations can further include generating the medical image based on the first information, the second information, and predetermined information. The predetermined information can be associated with images of anatomical objects of a type that is a type of the anatomical object.

## Claims

### Claim 1

A system configured to generate a medical image, the system comprising:
a neural network pre-trained with a set of images containing internal tissue of an anatomical object of a plurality of patients, the neural network including:
a pre-trained encoder configured to receive an input image and generate a content vector as an output, the content vector representing what features of the anatomical object are present in the input image; and
a pre-trained decoder configured to receive the content vector to generate an interpolated image as an output;
processing circuitry coupled to the neural network; and
memory coupled to the processing circuitry and having instructions stored therein that are executable by the processing circuitry to cause the system to perform operations including:
obtaining a first two-dimensional image of an anatomical object;
using the encoder, extracting first information associated with the anatomical object from the first image, the first information containing the content vector of the first image;
obtaining a second two-dimensional image of the anatomical object;
using the encoder, extracting second information associated with the anatomical object from the second image, the second information containing the content vector of the second image;
interpolating the first and second information to generate an interpolated content vector;
using the decoder, generating the medical image based on the interpolated content vector.

### Claim 2 (depends on 1)

The system of claim 1, wherein the first information and second information each include an attribute of a characteristic of the anatomical object, and wherein the predetermined information includes characteristics of the anatomical objects.

### Claim 3 (depends on 1)

The system of claim 1, the operations further comprising:
interpolated content vector and the predetermined information that is associated with images of anatomical objects of a type that is a type of the anatomical object.

### Claim 4 (depends on 3)

The system of claim 3, wherein the first image is an image of the anatomical object from a first position and a first pose,
wherein the second image is an image of the anatomical object from a second position and a second pose, the second position being different than the first position and/or the second pose being different than the first pose,
wherein interpolating includes:
generating a plurality of interpolations between the first information and second information, each interpolation of the plurality of interpolations being generated by assigning a different weight to the first information relative to the second information; and
selecting an interpolation of the plurality of interpolations as the interpolated content vector based on a feature of the anatomical object being represented better by the interpolation than another interpolation of the plurality of interpolations.

### Claim 5 (depends on 3)

The system of claim 3, wherein the predetermined information includes a globally-optimized style vector, the style vector representing where the features are located in the input image.

### Claim 6 (depends on 3)

The system of claim 3, the operations further comprising:
determining the predetermined information based on a feature of the object that is desired to be better illustrated in the medical image.

### Claim 7 (depends on 3)

The system of claim 3, wherein the predetermined information includes an indication of a way of segmenting anatomical objects, highlighting portions of the anatomical object, and labeling parts of the anatomical object, and
wherein the medical image includes a segmented version of the anatomical object, with portions of the anatomical object highlighted, and parts of the anatomical object labeled based on the predetermined information.

### Claim 8 (depends on 1)

The system of claim 1, further comprising:
an x-ray detector coupled to the processing circuitry,
wherein obtaining the first image includes capturing, by the x-ray detector, the first image, and
wherein obtaining the second image includes capturing, by the x-ray detector, the second image.

### Claim 9 (depends on 1)

The system of claim 1, the operations further comprising at least one of:
displaying the medical image;
providing the medical image to a robotic surgery system.

### Claim 10

A system configured to generate a medical image, the system comprising:
a neural network pre-trained with a set of non-optical images containing an anatomical object of a plurality of patients, the neural network including:
a pre-trained encoder configured to receive an input image and generate a content vector as an output, the content vector representing what features of the anatomical object are present in the input image; and
a pre-trained decoder configured to receive the content vector to generate an interpolated image as an output;
processing circuitry coupled to the neural network; and
memory storing the neutral network and coupled to the processing circuitry, the processing circuitry programmed to execute the following instructions:
obtaining a first non-optical image of an anatomical object containing internal features under a skin;
using the pre-trained encoder of the neural network, extracting the content vector associated with the anatomical object from the first image;
obtaining a second non-optical image of the anatomical object containing internal features under a skin;
using the pre-trained encoder of the neural network, extracting second information associated with the anatomical object from the second image, the second information containing the content vector of the second image;
interpolating the first and second information to generate an interpolated content vector;
using the pre-trained decoder of the neural network, generating the medical image based on the interpolated content vector.

### Claim 11 (depends on 10)

The system of claim 10, wherein the first and second images are 2-dimensional x-ray images and the generated medical image is an x-ray image which has been interpolated from the first and second x-ray images.

### Claim 12

The system of 11, the processing circuitry further programmed to execute the following instructions:
repeating the instructions of generating the medical image for M sets of first and second images to generate N medical images;
using the N sets of first and second images and N medical images, generating a 3-dimensional CT image for purposes of reducing x-ray exposure to the patient.