# US 11,992,277 B2 · Medical user interfaces and related methods of use

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,992,277 B2
- Document ID: US11992277B2
- Status: Granted
- Kind: B2
- Application number: 18/308,974
- Filed: 2023-04-28
- Published: 2023-08-17
- Granted: 2024-05-28
- Earliest claimed: 2016-01-29
- Assignees: Boston Scientific Scimed, Inc.
- Applicants: Boston Scientific Scimed, Inc.
- Inventors: Timothy P. Harrah; Christopher L. Oskin; Derrick Lenz; Arpita Banerjee; Sandesh Gavade; Pavan Misra; Abhijit Takale
- Web page: https://patentgrep.com/patent/US11992277B2

## CPC

- A61B 34/25: {User interfaces for surgical systems}
- A61B 18/26: for producing a shock wave, e.g. laser lithotripsy
- G06T 7/0014: {using an image reference approach}
- G06T 7/60: Analysis of geometric attributes
- G06T 11/65: on geographic maps
- G16H 40/63: for local operation
- A61B 6/464: {involving a plurality of displays}
- A61B 6/465: {adapted to display user selection data, e.g. graphical user interface, icons or menus}
- A61B 17/22004: {using mechanical vibrations, e.g. ultrasonic shock waves (A61B17/225 takes precedence)}
- A61B 2018/00505: {Urinary tract}
- A61B 2018/00511: {Kidney}
- A61B 2018/00517: {Urinary bladder or urethra}
- A61B 2018/00577: {Ablation}
- A61B 2018/00982: {combined with or comprising means for visual or photographic inspections inside the body, e.g. endoscopes}
- A61B 2034/107: {Visualisation of planned trajectories or target regions}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2034/2074: {Interface software}
- A61B 2034/252: {indicating steps of a surgical procedure}
- A61B 2034/254: {being adapted depending on the stage of the surgical procedure}
- A61B 2034/256: {having a database of accessory information, e.g. including context sensitive help or scientific articles}
- A61B 2217/007: with irrigation system
- A61B 2218/002: Irrigation
- G06T 2207/10116: X-ray image
- G06T 2207/30084: Kidney; Renal
- G06T 2210/41: Medical
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A medical system for use in a lithotripsy procedure may include a processor configured to receive input from a first imaging device, wherein the first imaging device may be configured to send image data representative of an image captured in a lumen of a kidney, bladder, or ureter to the processor. The processor may be configured to display the image on a display device coupled to the processor, and analyze the image to sense the presence of an object within the image. If an object was sensed within the image, the processor may analyze the image to estimate a size of the object, and display the estimate on the display device.

## Claims

### Claim 1

A medical system for use in a medical procedure, the medical system comprising:
an imaging device;
a display device; and
a processor configured to receive input from the imaging device, wherein the imaging device is configured to send image data representative of a first image captured within a body lumen of a patient to the processor, wherein the processor is coupled to the display device;
wherein the processor is configured to:
analyze the first image to sense a presence of an object within the first image;
when an object is sensed within the first image, analyze the first image to estimate a size of the object;
display, on the display device, a first user interface during a first portion of the medical procedure, wherein the first user interface includes the first image and the estimated size of the object;
display, on the display device, a second user interface during a second portion of the medical procedure, the second portion of the medical procedure including estimating a size of an orifice within a second image or an x-ray image, and wherein the second user interface includes the second image or the x-ray image and the estimated size of the orifice overlaid as a numerical value on either the second image or the x-ray image; and
display, on the display device, a third user interface during a third portion of the medical procedure, wherein the third portion of the medical procedure includes application of laser energy, and wherein the third user interface includes an image of an energy delivery element delivering laser energy to the object within the lumen;
wherein the processor is further configured to analyze the image of the energy delivery element delivering laser energy to the object within the lumen to sense whether fragments of the object have been removed by the laser energy; and
wherein the processor is configured to determine that a size of the object is decreasing based on a reduced proportion of a camera field being occupied by the object.

### Claim 2 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to update the estimated size of the object if the processor senses that fragments of the object have been removed by the laser energy.

### Claim 3 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to display an initial user interface before initiation of the medical procedure, wherein the initial user interface displays information regarding the patient, including at least one of a patient vitals field, a patient stats field, a diagnosis field, or an anatomy field.

### Claim 4 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to overlay the estimated size of the object as a numerical value on the first user interface.

### Claim 5 (depends on 1)

The medical system of claim 1, wherein the image data includes an image of a lumen of a kidney, a bladder, or a ureter.

### Claim 6 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to display, on the display device, another user interface during another portion of the medical procedure, wherein the another portion of the medical procedure includes delivering a stent into the lumen of the patient, and wherein the another user interface includes an image of the stent while delivered within the lumen of the patient.

### Claim 7 (depends on 1)

The medical system of claim 1, wherein the first user interface is not displayed during the second portion or the third portion of the medical procedure, wherein the second user interface is not displayed during the first portion or the third portion of the medical procedure, and wherein the third user interface is not displayed during the first portion or the second portion of the medical procedure.

### Claim 8 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to display, on the display device, an end user interface after the medical procedure has concluded, wherein the end user interface includes data relating to one or more of the first, second, or third portions of the medical procedure.

### Claim 9 (depends on 1)

The medical system of claim 1, wherein the processor is further configured to generate a scaled grid overlaid onto at least one of the first or second user interface, wherein the scaled grid is representative of an actual size of the object within the first image.

### Claim 10

A medical system for use in a medical procedure, the medical system comprising:
an imaging device;
a display device; and
a processor configured to receive input from the imaging device, wherein the imaging device is configured to send image data representative of a first image captured within a body lumen of a patient to the processor, wherein the processor is coupled to the display device;
wherein the processor is configured to:
analyze the first image to sense a presence of an object within the first image;
when an object is sensed within the first image, analyze the first image to estimate a size of the object;
display, on the display device, a first user interface during a first portion of the medical procedure, wherein the first user interface includes the first image and the estimated size of the object; and
display, on the display device, a second user interface during a second portion of the medical procedure, wherein the second portion of the medical procedure includes application of laser energy, and wherein the second user interface includes an image of an energy delivery element delivering laser energy to the object within the lumen;
wherein the processor is further configured to analyze the image of the energy delivery element delivering laser energy to the object within the lumen to sense whether fragments of the object have been removed by the laser energy and to estimate a size of one or more fragments of the object, and
wherein the object is a stone, wherein the second portion of the medical procedure is a lithotripsy procedure, and wherein the processor is configured to determine that a size of the stone is decreasing based on a reduced proportion of a camera field being occupied by the object.

### Claim 11 (depends on 10)

The medical system of claim 10, wherein the processor is configured to generate a scaled grid that is configured to be overlaid onto one or more of the first or second user interfaces, wherein the scaled grid is representative of an actual size of the object within the image.

### Claim 12 (depends on 10)

The medical system of claim 10, wherein the processor is further configured to display, on the display device, another user interface during another portion of the medical procedure, wherein the another portion of the medical procedure includes delivering a stent into the lumen of the patient.

### Claim 13 (depends on 12)

The medical system of claim 12, wherein the another user interface includes an image of the stent while delivered into the lumen of the patient.

### Claim 14 (depends on 10)

The medical system of claim 10, wherein the processor is further configured to display, on the display device, an end user interface after the medical procedure has concluded, wherein the end user interface includes data relating to at least one of the first or second portion of the medical procedure.

### Claim 15

A medical system for use in a medical procedure, the medical system comprising:
an imaging device;
a display device; and
a processor configured to receive input from the imaging device, wherein the imaging device is configured to send image data representative of an image captured in a body lumen of a patient to the processor, wherein the processor is coupled to the display device;
wherein the processor is configured to:
display the image on the display device;
analyze the image to sense a presence of one or more objects within the image;
when one or more objects are sensed within the image, analyze the image to estimate a size and/or a density of the one or more objects; and
update the estimate of the size and/or density of the one or more objects in real-time;
wherein the processor is further configured to update the estimate of the size and/or density of the one or more objects based on a proportion of each image that is occupied by the one or more objects.

### Claim 16 (depends on 15)

The medical system of claim 15, wherein the processor is configured, during application of laser energy during the medical procedure, to:
display an image of an energy delivery element delivering laser energy to the one or more objects; and
analyze one or more additional images to sense whether fragments of the one or more objects have been removed or broken up by the laser energy.

### Claim 17 (depends on 16)

The medical system of claim 16, wherein the one or more objects are one or more stones, and wherein the application of laser energy is a lithotripsy procedure.

### Claim 18 (depends on 15)

The medical system of claim 15, wherein the processor is further configured to display the estimate of the size and/or density of the one or more objects on the display device as a numerical value overlaid onto the image.

### Claim 19 (depends on 15)

The medical system of claim 15, wherein the processor is further configured to:
generate a scaled grid overlaid onto the image displayed on the display device, wherein the scaled grid is representative of an actual size of the one or more objects within the image; and
regenerate the scaled grid whenever the processor senses that the imaging device has been moved.