# US 2021/0275146 A1 · Apparatus and method for automatic ultrasound segmentation for visualization and measurement

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0275146 A1
- Document ID: US20210275146A1
- Status: Published application
- Kind: A1
- Application number: 16/813,469
- Filed: 2020-03-09
- Published: 2021-09-09
- Earliest claimed: 2020-03-09
- Assignees: Verdure Imaging, Inc.
- Applicants: Verdure Imaging, Inc.
- Inventors: Christopher Schlenger; Tamas Ungi
- Web page: https://patentgrep.com/patent/US20210275146A1

## CPC

- A61B 8/466: {adapted to display 3D data}
- A61B 8/5246: {combining images from the same or different imaging techniques, e.g. color Doppler and B-mode}
- A61B 5/1127: {using markers}
- A61B 8/0875: {for diagnosis of bone}
- A61B 8/4254: {using sensors mounted on the probe}
- A61B 8/4263: {using sensors not mounted on the probe, e.g. mounted on an external reference frame}
- A61B 8/463: {characterised by displaying multiple images or images and diagnostic data on one display}
- A61B 8/465: {adapted to display user selection data, e.g. icons or menus}
- A61B 8/468: {allowing annotation or message recording}
- A61B 8/483: {involving the acquisition of a 3D volume of data}
- A61B 8/5207: {involving processing of raw data to produce diagnostic data, e.g. for generating an image}
- A61B 8/5223: {for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012)}
- G06T 7/0016: {involving temporal comparison}
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- A61B 5/0077: {Devices for viewing the surface of the body, e.g. camera, magnifying lens}
- A61B 5/721: {using a separate sensor to detect motion or using motion information derived from signals other than the physiological signal to be measured}
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10136: 3D ultrasound image
- G06T 2207/30004: Biomedical image processing
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A system and method for performing ultrasound scans is provided. One embodiment of the ultrasonagraphic system acquires sonogram information from a series of ultrasonic scans of a human subject. The series of ultrasound scans are taken over a portion of interest on the human subject which has their underlying bone structure or other ultrasound discernable organ that is under examination. The data from the series of scans are synthesized into a single data file that corresponds to a three-dimensional (3D) image and/or 3D model of the underlying bone structure or organ of the examined human subject.

## Claims

### Claim 1

An ultrasonagraphic system, comprising:
an ultrasound transducer probe that is configured to acquire sonogram image information from a serial plurality of ultrasound scans of a human subject, wherein the ultrasound transducer probe generates the sonogram image information during each one of the plurality of ultrasound scans;
an ultrasound image data processor communicatively coupled to the ultrasound transducer probe,
wherein the ultrasound transducer probe receives the sonogram image information from the ultrasound transducer probe, and
wherein the ultrasound image data processor generates a plurality of time indexed sonogram image information portions that each include the sonogram image information and a first time that corresponds to a time of acquisition of the sonogram image information;
at least one image capture device that captures a time sequenced series of camera images that include both the human subject and the ultrasound transducer probe,
wherein at least one first optical target that is located on the human subject is visible in each one of the series of captured camera images, and
wherein at least one second optical target that is located on the ultrasound transducer probe is visible in each one of the series captured camera images;
an optical tracker unit communicatively coupled to the at least one image capture device,
wherein the optical tracker unit receives each one of the series of camera images from the at least one image capture device,
wherein each one of the series of camera images includes a time that indicates a time of acquisition of the camera image by the at least one image capture device;
wherein the optical tracker unit determines a location of the at least one first target on the human subject and a location of the at least one second target on the ultrasound transducer probe,
wherein the optical tracker unit determines time indexed location information for each camera image based on the determined locations of the first optical target and the second optical target,
wherein the time indexed location information identifies a portion of the human subject that is being scanned by the ultrasound transducer probe based on the determined location of the at least one optical target on the human subject and the at least one second optical target on the ultrasound transducer probe, and
wherein the time indexed location information includes the second time;
an image registration module that is communicatively coupled to the ultrasound image data processor and the at least one image capture device,
wherein the image registration module selects, for each time indexed sonogram image information portion, the time indexed location information that has the same second time as the first time, and
wherein the image registration module generates a plurality of location indexed sonogram information portions based on the each one of the time indexed sonogram image information portions and the corresponding time indexed location information; and
an image processing algorithm module that is communicatively coupled to the image registration module,
wherein the image processing algorithm module receives the plurality of location indexed sonogram image information portions from the image registration module, and
wherein the image processing algorithm module combines the received plurality of location indexed sonogram image information portions to generate a composite sonogram image information.

### Claim 2 (depends on 1)

The ultrasonagraphic system of claim 1, wherein the image processing algorithm module generates three dimensional (3D) composite sonogram graphical information based on the generated composite sonogram image information.

### Claim 3 (depends on 2)

The ultrasonagraphic system of claim 2, further comprising:
a 3D visualization module that generates an image based on the 3D or composite sonogram graphical information,
wherein the image is presentable on a display.

### Claim 4 (depends on 1)

The ultrasonagraphic system of claim 1, further comprising:
a clock that provides the time of acquisition of the ultrasound information by the ultrasound transducer probe,
wherein the clock resides in the ultrasound image data processor.

### Claim 5 (depends on 1)

The ultrasonagraphic system of claim 1, further comprising:
a clock that provides the time of acquisition of the camera image that is captured by the at least one image capture device,
wherein the clock resides in the ultrasound image data processor.

### Claim 6 (depends on 1)

The ultrasonagraphic system of claim 1,
wherein the ultrasound transducer probe and the at least one image capture device are located in a first location where the human subject is being examined,
wherein at least the optical tracker unit, the image registration module, and the image processing algorithm module reside in a second location that is remote from the first location, and
wherein the optical tracker unit, the image registration module, and the image processing algorithm module are operable to receive information form a plurality Preliminary Amendment Page 4 of 9 of other ultrasound transducer probes and the associated at least one image capture device at different first locations where different human subjects are being examined.

### Claim 7

A method of examining a human subject using an ultrasonagraphic system, comprising:
receiving a serial plurality of ultrasound scans of the human subject from the ultrasound transducer probe;
generating a plurality of time indexed sonogram image information portions that each include sonogram image information from a portion of the ultrasound scan and a first time that corresponds to a time of acquisition of the sonogram image information portion;
capturing a time sequenced series of camera images that include both the human subject and the ultrasound transducer probe using at least one image capture device, wherein each one of the series of camera images includes a second time that indicates a time of acquisition of the camera image by the at least one image capture device;
determining time indexed location information for each camera image,
wherein the time indexed location information identifies a location of a portion of the human subject that is being scanned by the ultrasound transducer probe, and
wherein the time indexed location information includes the associated second time;
selecting, for each time indexed sonogram image information portion, the time indexed location information that has the same second time as the first time of the sonogram image information portion;
generating a plurality of location indexed sonogram information portions based on the each one of the time indexed sonogram image information portions and the corresponding time indexed location information; and
combining the received plurality of location indexed sonogram image information portions to generate a composite sonogram image information.

### Claim 8 (depends on 7)

The method of claim 7,
wherein at least one first optical target is located on the human subject and is visible in each one of the series of captured camera images,
wherein at least one second optical target that is located on the ultrasound transducer probe is visible in each one of the series captured camera images,
wherein determining time indexed location information for each camera image comprises:
determining a location of the at least one first target on the human subject and a location of the at least one second target on the ultrasound transducer probe, and
wherein the location of the portion of the human subject that is being scanned by the ultrasound transducer probe is determined based on the determined location of the at least one first target and the determined location of the at least one second target.

### Claim 9 (depends on 7)

The method of claim 7, wherein generating a plurality of time indexed sonogram image information portions comprises:
adding the first time to the each one of the time indexed sonogram image information portions using a clock of an ultrasound image data processor that receives the serial plurality of ultrasound scans from the remotely located ultrasound transducer probe.

### Claim 10 (depends on 7)

The method of claim 7, wherein after capturing a time sequenced series of camera images, the method comprises:
adding the second time to the each one of the camera images using a clock of an ultrasound image data processor that receives the serial plurality of ultrasound scans from the remotely located ultrasound transducer probe.

### Claim 11 (depends on 7)

The method of claim 7, further comprising:
generating three dimensional (3D) composite sonogram graphical information based on the generated composite sonogram image information.

### Claim 12 (depends on 11)

The method of claim 11,
generating a 3D image based on the 3D composite sonogram graphical information; and
presenting the 3D image on a display.

### Claim 13 (depends on 12)

The method of claim 12,
wherein 3D image is generated and presented on the display after all of the serial plurality of ultrasound scans of the human subject have been received from the ultrasound transducer probe.

### Claim 14 (depends on 12)

The method of claim 12,
wherein 3D image is generated and presented on the display as each one of the serial plurality of ultrasound scans of the human subject is being received from the ultrasound transducer probe.

### Claim 15 (depends on 11)

The method of claim 11,
generating a 2D image based on the 3D composite sonogram graphical information; and
presenting the 2D image on the display.

### Claim 16 (depends on 7)

The method of claim 7,
filtering the sonogram image information in each received portion of the ultrasound scan to discriminate an organ of interest from other surrounding organs,
wherein sonogram image information associated with the other surrounding organs is removed.

### Claim 17 (depends on 16)

The method of claim 16, wherein the organ of interest is a spine of the human subject.

### Claim 18 (depends on 16)

The method of claim 16, further comprising
analyzing the image of the spine to determine at least one of position and orientation between two vertebrae of the spine.

### Claim 19 (depends on 18)

The method of claim 18, wherein analyzing the image of the spine to determine at least one of the position and the orientation between two vertebrae of the spine is performed using an artificial intelligence algorithm that improves the filtering such that discrimination between the bone and the surrounding organs is improved as a plurality of different human subjects are examined.

### Claim 20 (depends on 7)

The method of claim 7, wherein generating the 3D composite sonogram graphical information comprises:
generating a graphical artifact that indicates the determined at least one of position and orientation between two vertebrae of the spine; and
presenting the generated graphical artifact on the presented image of the spine.
