# US 2022/0265121 A1 · Systems and methods for controlling a surgical pump using endoscopic video data

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0265121 A1
- Document ID: US20220265121A1
- Status: Published application
- Kind: A1
- Application number: 17/651,412
- Filed: 2022-02-16
- Published: 2022-08-25
- Earliest claimed: 2021-02-25
- Assignees: Stryker Corporation
- Applicants: Stryker Corporation
- Inventors: Brian FOUTS; Cole Kincaid HUNTER; Brady WOOLFORD; Amit A. MAHADIK; Hannes RAU; Wenjing LI; Joel M. ERNST
- Web page: https://patentgrep.com/patent/US20220265121A1

## CPC

- A61B 1/00006: {of control signals}
- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- A61B 1/015: Control of fluid supply or evacuation
- A61B 1/317: for bones or joints, e.g. osteoscopes or arthroscopes
- A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61M 1/77: {Suction-irrigation systems (aspiration tips supplying fluids A61M1/85; specific for negative pressure wound therapy A61M1/92; combined with tracheal tubes A61M16/0463)}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/0016: {involving temporal comparison}
- G06V 10/764: using classification, e.g. of video objects
- A61B 2017/00203: {with speech control or speech recognition}
- A61B 2034/2055: {Optical tracking systems}
- A61M 1/74: {Suction control (underwater drainage A61M1/61)}
- A61M 2205/6063: Optical identification systems
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10068: Endoscopic image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30008: Bone
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61M: DEVICES FOR INTRODUCING MEDIA INTO, OR ONTO, THE BODY (introducing media into or onto the bodies of animals A61D7/00; means for inserting tampons A61F13/26; devices for administering food or medicines orally A61J; containers for collecting, storing or administering blood or medical fluids A61J1/05); DEVICES FOR TRANSDUCING BODY MEDIA OR FOR TAKING MEDIA FROM THE BODY (surgery A61B; chemical aspects of surgical articles A61L); DEVICES FOR PRODUCING OR ENDING SLEEP OR STUPOR {(Electrotherapy, e.g. producing anaesthesia by the use of alternating or intermittent currents A61N1/36021)}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

According to an aspect, video data taken from an endoscopic imaging device can be used to automatically control a surgical pump for purposes of regulating fluid pressure in an internal area of a patient during an endoscopic procedure. Control of the pump can be based in part on one or more features extracted from video data received from an endoscopic imaging device. The features can be extracted from the video data using a combination of machine learning classifiers and other processes configured to determine the presence of various conditions within the images of the internal area of the patient. Using the one or more extracted features, the system can adjust the inflow and outflow settings of the surgical pump to regulate the fluid pressure of the internal area of the patient commensurate with the needs of the surgery and the patient at any given moment in time during the surgical procedure.

## Claims

### Claim 1

A method for controlling a fluid pump for use in surgical procedures, the method comprising:
receiving video data captured from an imaging tool configured to image an internal portion of a patient;
applying one or more machine learning classifiers to the received video data to generate one or more classification metrics based on the received video data, wherein the one or more machine learning classifiers are created using a supervised training process that comprises using one or more annotated images to train the machine learning classifier;
determining the presence of one or more conditions in the received video data based on the generated one or more classification metrics; and
determining an adjusted setting for the flow through or head pressure from the fluid pump based on the determined presence of the one or more conditions in the received video data.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more machine learning classifiers comprises a joint type machine learning classifier configured to generate one or more classification metrics associated with identifying a type of joint pictured in the received video data.

### Claim 3 (depends on 2)

The method of claim 2, wherein the joint type machine learning classifier is configured to identify one or more joints selected from the group consisting of a hip, a shoulder, a knee, an ankle, a wrist, and an elbow.

### Claim 4 (depends on 3)

The method of claim 3, wherein the joint type machine learning classifier is configured to generate one or more classification metrics associated with identifying whether the imaging tool is not within a joint.

### Claim 5 (depends on 4)

The method of claim 4, wherein the one or more machine learning classifiers include a procedure stage machine learning classifier configured to generate one or more classification metrics associated with identifying a procedure stage being performed in the received video data.

### Claim 6 (depends on 1)

The method of claim 1, wherein the one or more machine learning classifiers comprises an instrument identification machine classifier configured to generate one or more classification metrics associated with identifying one or more instruments in the received video data.

### Claim 7 (depends on 6)

The method of claim 6, wherein the instrument identification machine classifier is configured to identify instruments selected from the group consisting of a shaver tool, a radio frequency (RF) probe, and a dedicated suction device.

### Claim 8 (depends on 6)

The method of claim 6, wherein the fluid pump is configured to activate a suction functionality of the one or more instruments based on the one or more classification metrics generated by the instrument identification machine classifier.

### Claim 9 (depends on 1)

The method of claim 1, wherein the one or more machine learning classifiers include an image clarity machine learning classifier configured to generate one or more classification metrics associated with a clarity of the received video data.

### Claim 10 (depends on 9)

The method of claim 9, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of blood visible in the received video data.

### Claim 11 (depends on 9)

The method of claim 9, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of bubbles visible in the received video data.

### Claim 12 (depends on 9)

The method of claim 9, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of debris visible in the received video data.

### Claim 13 (depends on 9)

The method of claim 9, wherein determining the presence of one or more conditions in the received video data based on the generated one or more classification metrics comprises determining if a clarity of the video is above a pre-determined threshold, and wherein the determination is based on the one more classification metrics generated by the image clarity machine classifier.

### Claim 14

A system for controlling a fluid pump for use in surgical procedures, the system comprising:
a memory;
one or more processors;
wherein the memory stores one or more programs that when executed by the one or more processors, cause the one or more processors to:
receive video data captured from an imaging tool configured to image an internal portion of a patient;
apply one or more machine learning classifiers to the received video data to generate one or more classification metrics based on the received video data, wherein the one or more machine learning classifiers are created using a supervised training process that comprises using one or more annotated images to train the machine learning classifier;
determine the presence of one or more conditions in the received video data based on the generated one or more classification metrics; and
adjust the flow through or head pressure from the fluid pump based on the determined presence of the one or more conditions in the received video data.

### Claim 15 (depends on 14)

The system of claim 14, wherein the one or more machine learning classifiers comprises a joint type machine learning classifier configured to generate one or more classification metrics associated with identifying a type of joint pictured in the received video data.

### Claim 16 (depends on 15)

The system of claim 15, wherein the joint type machine learning classifier is configured to identify one or more joints selected from the group consisting of a hip, a shoulder, a knee, an ankle, a wrist, and an elbow.

### Claim 17 (depends on 16)

The system of claim 16, wherein the joint type machine learning classifier is configured to generate one or more classification metrics associated with identifying whether the imaging tool is not within a joint.

### Claim 18 (depends on 17)

The system of claim 17, wherein the one or more machine learning classifiers include a procedure stage machine learning classifier configured to generate one or more classification metrics associated with identifying a procedure stage being performed in the received video data.

### Claim 19 (depends on 14)

The system of claim 14, wherein the one or more machine learning classifiers comprises an instrument identification machine classifier configured to generate one or more classification metrics associated with identifying one or more instruments in the received video data.

### Claim 20 (depends on 19)

The system of claim 19, wherein the instrument identification machine classifier is configured to identify instruments selected from the group consisting of a shaver tool, a radio frequency (RF) probe, and a dedicated suction device.

### Claim 21 (depends on 19)

The system of claim 19, wherein the fluid pump is configured to activate a suction functionality of the one or more instruments based on the one or more classification metrics generated by the instrument identification machine classifier.

### Claim 22 (depends on 14)

The system of claim 14, wherein the one or more machine learning classifiers include an image clarity machine learning classifier configured to generate one or more classification metrics associated with a clarity of the received video data.

### Claim 23 (depends on 22)

The system of claim 22, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of blood visible in the received video data.

### Claim 24 (depends on 22)

The system of claim 22, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of bubbles visible in the received video data.

### Claim 25 (depends on 22)

The system of claim 22, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of debris visible in the received video data.

### Claim 26 (depends on 22)

The system of claim 22, wherein determining the presence of one or more conditions in the received video data based on the generated one or more classification metrics comprises determining if a clarity of the video is above a pre-determined threshold, and wherein the determination is based on the one more classification metrics generated by the image clarity machine classifier.

### Claim 27

A non-transitory computer readable storage medium storing one or more programs for controlling a fluid pump for use in surgical procedures, for execution by one or more processors of an electronic device that when executed by the device, cause the device to:
receive video data captured from an imaging tool configured to image an internal portion of a patient;
apply one or more machine learning classifiers to the received video data to generate one or more classification metrics based on the received video data, wherein the one or more machine learning classifiers are created using a supervised training process that comprises using one or more annotated images to train the machine learning classifier;
determine the presence of one or more conditions in the received video data based on the generated one or more classification metrics; and
adjust the flow through or head pressure from the fluid pump based on the determined presence of the one or more conditions in the received video data.

### Claim 28 (depends on 27)

The non-transitory computer readable storage medium of claim 27, wherein the one or more machine learning classifiers comprises a joint type machine learning classifier configured to generate one or more classification metrics associated with identifying a type of joint pictured in the received video data.

### Claim 29 (depends on 28)

The non-transitory computer readable storage medium of claim 28, wherein the joint type machine learning classifier is configured to identify one or more joints selected from the group consisting of a hip, a shoulder, a knee, an ankle, a wrist, and an elbow.

### Claim 30 (depends on 29)

The non-transitory computer readable storage medium of claim 29, wherein the joint type machine learning classifier is configured to generate one or more classification metrics associated with identifying whether the imaging tool is not within a joint.

### Claim 31 (depends on 30)

The non-transitory computer readable storage medium of claim 30, wherein the one or more machine learning classifiers include a procedure stage machine learning classifier configured to generate one or more classification metrics associated with identifying a procedure stage being performed in the received video data.

### Claim 32 (depends on 27)

The non-transitory computer readable storage medium of claim 27, wherein the one or more machine learning classifiers comprises an instrument identification machine classifier configured to generate one or more classification metrics associated with identifying one or more instruments in the received video data.

### Claim 33 (depends on 32)

The non-transitory computer readable storage medium of claim 32, wherein the instrument identification machine classifier is configured to identify instruments selected from the group consisting of a shaver tool, a radio frequency (RF) probe, and a dedicated suction device.

### Claim 34 (depends on 32)

The non-transitory computer readable storage medium of claim 32, wherein the fluid pump is configured to activate a suction functionality of the one or more instruments based on the one or more classification metrics generated by the instrument identification machine classifier.

### Claim 35 (depends on 27)

The non-transitory computer readable storage medium of claim 27, wherein the one or more machine learning classifiers include an image clarity machine learning classifier configured to generate one or more classification metrics associated with a clarity of the received video data.

### Claim 36 (depends on 35)

The non-transitory computer readable storage medium of claim 35, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of blood visible in the received video data.

### Claim 37 (depends on 35)

The non-transitory computer readable storage medium of claim 35, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of bubbles visible in the received video data.

### Claim 38 (depends on 35)

The s non-transitory computer readable storage medium of claim 35, wherein the image clarity machine classifier is configured to generate one or more classification metrics associated with an amount of debris visible in the received video data.

### Claim 39 (depends on 35)

The non-transitory computer readable storage medium of claim 35, wherein determining the presence of one or more conditions in the received video data based on the generated one or more classification metrics comprises determining if a clarity of the video is above a pre-determined threshold, and wherein the determination is based on the one more classification metrics generated by the image clarity machine classifier.
