# US 12,178,651 B2 · Real-time anatomically based deformation mapping and correction

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,178,651 B2
- Document ID: US12178651B2
- Status: Granted
- Kind: B2
- Application number: 17/304,245
- Filed: 2021-06-17
- Published: 2022-12-22
- Granted: 2024-12-31
- Earliest claimed: 2016-05-16
- Assignees: BK Medical Holding Company, Inc.
- Applicants: BK Medical Holding Company, Inc.
- Inventors: David Lieblich; Zhaolin Li
- Web page: https://patentgrep.com/patent/US12178651B2

## CPC

- A61B 8/483: {involving the acquisition of a 3D volume of data}
- A61B 8/5223: {for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012)}
- A61B 8/08: Clinical applications (A61B8/02, A61B8/04, A61B8/06 take precedence)
- A61B 8/485: {involving measuring strain or elastic properties}
- A61B 8/5207: {involving processing of raw data to produce diagnostic data, e.g. for generating an image}
- A61B 8/5246: {combining images from the same or different imaging techniques, e.g. color Doppler and B-mode}
- G06T 7/50: Depth or shape recovery
- G06T 7/64: of convexity or concavity
- G06T 7/73: using feature-based methods
- G06T 12/10: Image preprocessing, e.g. calibration, positioning of sources or scatter correction
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2207/10132: Ultrasound image
- G06T 2207/10136: 3D ultrasound image
- G06T 2219/2021: Shape modification
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method includes generating a real-time ultrasound image of anatomy of interest. At least a sub-portion of the anatomy of interest is deformed from an initial location to a different location by pressure applied by an external force. The method further includes obtaining a 2-D slice, which corresponds to a same plane as the real-time ultrasound image, from 3-D reference image data, wherein a corresponding sub-portion is at the initial location. The method further includes determining displacement fields for the sub-portion from the sub-portion, the corresponding sub-portion and other anatomy not-deformed in the real-time ultrasound image and the 3-D reference image data. The method further includes deforming the 3-D reference image data using the displacement fields, which creates deformed 3-D reference image data based on the different location.

## Claims

### Claim 1

A method, comprising:
generating a real-time ultrasound image of an anatomy of interest of a subject being scanned, wherein at least a first sub-portion of the anatomy of interest is deformed from an initial location to a different location by pressure applied by an external force and a second sub-portion of the anatomy of interest is not deformed by the pressure applied by the external force;
obtaining, during a scan, a 2-D slice, which corresponds to a same plane as the real-time ultrasound image, from previously acquired 3-D reference image data, wherein corresponding first and second sub-portions are at the initial location in the 2-D slice;
measuring a first distance between a contour of the first sub-portion and a contour of the second sub-portion in the real-time ultrasound image;
measuring a second distance between a contour of the first sub-portion and a contour of the second sub-portion in the 2-D slice;
determining a differential displacement of the first sub-portion from the first distance and the second distance; and
deforming, during the scan, the previously acquired 3-D reference image data based on the differential displacement to create deformed 3-D reference image data;
combining the real-time ultrasound image with the deformed 3-D reference image data at a corresponding plane in the previously acquired 3-D reference image data; and
displaying the combined data, wherein the combined data is used to navigate a probe to a tissue of interest in the anatomy of interest.

### Claim 2 (depends on 1)

The method of claim 1, wherein deforming the 3-D reference image data includes re-positioning the first sub-portion in the 3-D previously acquired reference image data at a location in the 3-D reference image data that corresponds to the different location.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
identifying the contour of the first sub-portion in the real-time ultrasound image; identifying the contour of the second sub-portion in the real-time ultrasound image;
identifying the contour of the first sub-portion in the 2-D slice; and identifying the contour of the second sub-portion in the 2-D slice.

### Claim 4 (depends on 1)

The method of claim 1, wherein the real-time ultrasound image and the previously acquired 3-D reference image data include an undeformed structure, and further comprising:
determining the differential displacement based on the undeformed structure.

### Claim 5 (depends on 1)

The method of claim 1, wherein the differential displacement represents an indentation along, at least, a finite length of an image plane.

### Claim 6 (depends on 1)

The method of claim 1, wherein the deforming of the previously acquired 3-D reference image data includes mapping anatomical structures within the previously acquired 3-D reference image data based on the differential displacement.

### Claim 7 (depends on 1)

The method of claim 1, wherein the deforming of the previously acquired 3-D reference image data includes employing directional derivatives of the differential displacement relative to a maximum displacement vector of a probe plane.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
using one of cylindrical symmetry of the directional derivatives and a smooth transition between two differing derivatives orthogonal to a single plane.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
deriving gradients for intervening directions using gradients in two orthogonal half planes for two planes with differing gradients.

### Claim 10

A non-transitory computer readable medium encoded with computer executable instructions, which, when executed by a computer processor, causes the processor to:
generate a real-time ultrasound image of an anatomy of interest of a subject being scanned, wherein at least a first sub-portion of the anatomy of interest is deformed from an initial location to a different location by pressure applied by an external force and a second sub-portion of the anatomy of interest is not deformed by the pressure applied by the external force;
obtain, during a scan, a 2-D slice, which corresponds to a same plane as the real-time ultrasound image, from previously acquired 3-D reference image data, wherein corresponding first and second sub-portions are at the initial location in the 2-D slice;
measure a first distance between a contour of the first sub-portion and a contour of the second sub-portion in the real-time ultrasound image;
measure a second distance between a contour of the first sub-portion and a contour of the second sub-portion in the 2-D slice;
determine a differential displacement of the first sub-portion from the first distance and the second distance; and
deform, during the scan, the previously acquired 3-D reference image data based on the differential displacement to create deformed 3-D reference image data;
combine the real-time ultrasound image with the deformed 3-D reference image data at a corresponding plane in the previously acquired 3-D reference image data; and
display the combined data, wherein the combined data is used to navigate a probe to a tissue of interest in the anatomy of interest.

### Claim 11 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the computer executable instructions further cause the processor to:
determine the differential displacement based on undeformed structures.

### Claim 12 (depends on 10)

The non-transitory computer readable medium of claim 10, wherein the computer executable instructions further cause the processor to:
deform the previously acquired 3-D reference image data using directional derivatives of the differential displacement relative to a maximum displacement vector of a probe plane.

### Claim 13 (depends on 12)

The non-transitory computer readable medium of claim 12, wherein the computer executable instructions further cause the processor to:
derive gradients for intervening directions using gradients in two orthogonal half planes for two planes with differing gradients.

### Claim 14

An apparatus, comprising:
a beamformer configured to generate a real-time ultrasound image of an anatomy of interest of a subject being scanned, wherein at least a first sub-portion of the anatomy of interest is deformed from an initial location to a different location by pressure applied by an external force and a second sub-portion of the anatomy of interest is not deformed by the pressure applied by the external force;
a 3-D reference image data memory that stores 3-D reference image data of a volume of interest; and
a processor configured to:
obtain, during a scan, a 2-D slice, which corresponds to a same plane as the real-time ultrasound image, from the stored 3-D reference image data, wherein corresponding first and second sub-portions in the stored 3-D reference image data are at the initial location;
measure a first distance between a contour of the first sub-portion and a contour of the second sub-portion in the real-time ultrasound image;
measure a second distance between a contour of the first sub-portion and a contour of the second sub-portion in the 2-D slice;
determine a differential displacement of the first sub-portion from the first distance and the second distance; and
deform, during the scan, the stored 3-D reference image data based on the differential displacement to create deformed 3-D reference image data;
combine the real-time ultrasound image with the deformed 3-D reference image data at a corresponding plane in the previously acquired 3-D reference image data; and
display the combined data, wherein the combined data is used to navigate a probe to a tissue of interest in the anatomy of interest.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the processor warps the stored 3-D reference image data to create the deformed 3-D reference image data.

### Claim 16 (depends on 14)

The apparatus of claim 14, wherein the processor deforms the stored 3-D reference image data based on a decay of the displacement.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the decay is represented by (∂U→_y/∂z _+, where the “+” sign indicates a positive coordinate axis direction of a coordinate system tied to the center of a hemispherical cap and with a different axis intersecting an image plane.

### Claim 18 (depends on 14)

The apparatus of claim 14, wherein the processor determines the differential displacement based only on the contour of the first sub-portion in the real-time ultrasound image, the contour of the first sub-portion in the 2-D slice, the contour of the second sub-portion in the real-time ultrasound image, and the contour of the second sub-portion in the 2-D slice.