# US 12,178,637 B2 · Image fusion-based tracking without a tracking sensor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,178,637 B2 - Document ID: US12178637B2 - Status: Granted - Kind: B2 - Application number: 17/024,954 - Filed: 2020-09-18 - Published: 2022-03-24 - Granted: 2024-12-31 - Earliest claimed: 2020-09-18 - Assignees: BK Medical Aps - Applicants: B-K Medical ApS - Inventors: Jens Munk Hansen; Henrik Jensen; Jacob Bjerring Olesen; Fredrik Gran; Bo Martins - Web page: https://patentgrep.com/patent/US12178637B2 ## CPC - A61B 8/0841: {for locating instruments} - A61B 8/4245: {involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient} - A61B 8/4416: {related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions} - A61B 8/483: {involving the acquisition of a 3D volume of data} - A61B 8/5207: {involving processing of raw data to produce diagnostic data, e.g. for generating an image} - A61B 8/5246: {combining images from the same or different imaging techniques, e.g. color Doppler and B-mode} - A61B 8/5253: {combining overlapping images, e.g. spatial compounding} - A61B 8/5261: {combining images from different diagnostic modalities, e.g. ultrasound and X-ray} - G01S 7/52026: {Extracting wanted echo signals (Doppler systems G01S15/50; Doppler short range imaging systems G01S15/8979)} - G01S 7/52074: {Composite displays, e.g. split-screen displays; Combination of multiple images or of images and alphanumeric tabular information} - G01S 15/89: for mapping or imaging - G01S 15/8915: {using a transducer array} - G01S 15/8993: {Three dimensional imaging systems} - A61B 5/4381: {Prostate evaluation or disorder diagnosis} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract An ultrasound imaging system include a console with transmit circuitry configured to generate an excitation electrical pulse that excites transducer elements of a probe to produce an ultrasound pressure field and configured to receive an echo signal generated in response to the ultrasound pressure field interacting with tissue. The console further includes receive circuitry configured to convert the echo signal into an electrical signal, and an echo processor configured to generate a live ultrasound image based on the electrical signal into. The console further includes a tracking processor configured to extract, in real-time, a feature common to both the live ultrasound image and previously generated volumetric image data from at least the live ultrasound image, and register, in real-time, the live ultrasound image with previously generated volumetric image data based on the common feature extracted in real-time to create the fused image. A display displays the fused image. ## Claims ### Claim 1 An ultrasound imaging system, comprising: a console, including: transmit circuitry configured to generate an excitation electrical pulse that excites transducer elements of a probe to produce an ultrasound pressure field and configured to receive an echo signal generated in response to the ultrasound pressure field interacting with tissue; receive circuitry configured to convert the echo signal into an electrical signal; an echo processor configured to generate a live ultrasound image each frame based on the electrical signal; a tracking processor configured to: obtain the live ultrasound image and volumetric image data that was previously generated by another imaging system, which was configured for three-dimensional imaging; extract, in real-time, an anatomical feature common to both the live ultrasound image and the previously generated volumetric image data from at least the live ultrasound image; and register, in real-time, the live ultrasound image with the volumetric image data based on the common anatomical feature extracted in real-time to create a fused image; wherein the tracking processor registers the live ultrasound image with the volumetric image data based on image registration transformation parameters, updates the image registration transformation parameters based on a latest result of the registration and an earlier result of the registration and applies a filter removing movement outliers; and a display configured to display the fused image. ### Claim 2 (depends on 1) The ultrasound imaging system of claim 1, wherein the tracking processor fuses the live image and the previously generated volumetric image data without information from a positioning system to provide a relative position between the live image and the volumetric image data. ### Claim 3 (depends on 1) The ultrasound imaging system of claim 1, wherein the tracking processor continually fuses, in real-time, newly generated live ultrasound images with the volumetric image data, thereby continually creating updated fused images. ### Claim 4 (depends on 3) The ultrasound imaging system of claim 3, wherein the tracking processor continually fuses only a predetermined sub-set of the newly generated live ultrasound images. ### Claim 5 (depends on 4) The ultrasound imaging system of claim 4, wherein the tracking processor spatially tracks a biopsy instrument during an ultrasound-guided transrectal prostrate biopsy procedure performed with the ultrasound imaging system based on the fused images. ### Claim 6 (depends on 1) The ultrasound imaging system of claim 1, wherein the tracking processor, prior to receiving the live ultrasound image: receives an initial live ultrasound image, extracts, in real-time, the common feature from an initial live ultrasound image, registers the initial live ultrasound image with the volumetric image data based on the common feature extracted, and initializes parameters of an image transformation. ### Claim 7 (depends on 6) The ultrasound imaging system of claim 6, wherein the tracking processor updates the parameters of the image transformation after each new registration. ### Claim 8 (depends on 1) The ultrasound imaging system of claim 1, wherein the live ultrasound image is a two-dimensional image or a three-dimensional image. ### Claim 9 (depends on 1) The ultrasound imaging system of claim 1, wherein the registration is dynamically updated based on a feature visible in the live ultrasound image. ### Claim 10 (depends on 1) The ultrasound imaging system of claim 1, further comprising: a positioning system for providing a relative position between a first live image and the volumetric image data. ### Claim 11 (depends on 10) The ultrasound imaging system of claim 10, wherein the first live ultrasound image is registered and fused with the volumetric image data and the registration of the first live ultrasound image and the volumetric image is adjusted to re-align the fused images based on an extracted feature that is common to both the first live ultrasound image and the volumetric image data. ### Claim 12 (depends on 10) The ultrasound imaging system of claim 10, wherein the console receives a user input indicating the registration and fusing of the first live ultrasound image and the previously generated volumetric image data that are based on the relative position provided by the positioning system is incorrect, and the input invokes the tracking processor to perform the re-alignment of the fused images. ### Claim 13 (depends on 1) The ultrasound imaging system of claim 1, wherein the tracking processor is configured to perform the registration by formulating a cost function. ### Claim 14 (depends on 13) The ultrasound imaging system of claim 13, wherein the cost function depends on a location of the common feature in the live ultrasound image and the volumetric image data. ### Claim 15 (depends on 13) The ultrasound imaging system of claim 13, wherein the cost function includes a numerical optimization algorithm. ### Claim 16 (depends on 6) The ultrasound imaging system of claim 6, wherein the tracking processor validates the registration. ### Claim 17 (depends on 16) The ultrasound imaging system of claim 16, wherein the tracking processor validates the registration based on a value of a distance function for the registration. ### Claim 18 (depends on 1) The ultrasound imaging system of claim 1, wherein the anatomical feature is at least one of in and around the anatomy of interest. ### Claim 19 An ultrasound imaging system, comprising: a console, including: transmit circuitry configured to generate an excitation electrical pulse that excites transducer elements of a probe to produce an ultrasound pressure field and configured to receive an echo signal generated in response to the ultrasound pressure field interacting with tissue; receive circuitry configured to convert the echo signal into an electrical signal; an echo processor configured to generate a live ultrasound image for each frame based on the electrical signal; a tracking processor configured to: obtain the live ultrasound image and volumetric image data that was previously generated by another imaging system, which was configured for three-dimensional imaging; extract, in real-time, an anatomical feature common to both the live ultrasound image and the previously generated volumetric image data from at least the live ultrasound image; and perform a single registration, in real-time, of the live ultrasound image with the volumetric image data based on the common anatomical feature extracted in real-time to create a fused image; wherein the tracking processor registers the live ultrasound image with the volumetric image data based on image registration transformation parameters, updates the image registration transformation parameters based on a latest result of the registration and an earlier result of the registration and applies a filter removing movement outliers, and a display configured to display the fused image.