# US 2021/0275003 A1 · System and method for generating a three-dimensional model of a surgical site

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0275003 A1
- Document ID: US20210275003A1
- Status: Published application
- Kind: A1
- Application number: 17/329,029
- Filed: 2021-05-24
- Published: 2021-09-09
- Earliest claimed: 2018-12-10
- Assignees: Covidien LP
- Applicants: Covidien LP
- Inventors: John W. Komp
- Web page: https://patentgrep.com/patent/US20210275003A1

## CPC

- A61B 1/3132: {for laparoscopy}
- A61B 1/000094: {extracting biological structures}
- A61B 1/00172: {with means for scanning}
- A61B 1/00193: {adapted for stereoscopic vision}
- A61B 1/00194: {adapted for three-dimensional imaging}
- A61B 1/2676: {Bronchoscopes}
- A61B 1/31: for the rectum, e.g. proctoscopes or sigmoidoscopes
- G06T 7/11: Region-based segmentation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/003: {Navigation within 3D models or images}
- G16H 30/40: for processing medical images, e.g. editing
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system for generating a 3D model of a surgical site includes a 3D endoscope and a computing device coupled to the 3D endoscope. The 3D endoscope includes a scanner for scanning a surface of a surgical site and a camera source for generating images of the surgical site. A 3D model of the surgical site, including objects therein, is generated using scan data and image data. The 3D model is updated by detecting a change in the surgical site, isolating a region of the surgical site where the change is detected, generating second scan data by scanning the surface of the isolated region, and updating the 3D model generated using the second scan data of the surface of the isolated region.

## Claims

### Claim 1

(canceled)

### Claim 2

A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning from a source separate from the endoscope;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in the isolated regions of the surgical site.

### Claim 3 (depends on 2)

The method of claim 2, wherein source separate from the endoscope is a motion sensor on a patient or a separate camera.

### Claim 4 (depends on 2)

The method of claim 2, wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

### Claim 5 (depends on 2)

The method of claim 2 further comprising capturing optical images of the surgical site.

### Claim 6 (depends on 5)

The method of claim 5, wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

### Claim 7 (depends on 5)

The method of claim 5, wherein the optical images and the images of the scan pattern are simultaneously captured.

### Claim 8 (depends on 7)

The method of claim 7, wherein further comprising correlation of the optical images with preoperative images.

### Claim 9

A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in just the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in just the isolated regions of the surgical site.

### Claim 10 (depends on 9)

The method of claim 9, wherein detecting the change in the surgical site includes detecting movement of an object in the surgical site.

### Claim 11 (depends on 9)

The method of claim 9, wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

### Claim 12 (depends on 9)

The method of claim 9 further comprising capturing optical images of the surgical site.

### Claim 13 (depends on 12)

The method of claim 12, wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

### Claim 14 (depends on 12)

The method of claim 12, wherein the optical images and the images of the scan pattern are simultaneously captured.

### Claim 15 (depends on 14)

The method of claim 14, wherein further comprising correlation of the optical images with preoperative images.

### Claim 16

A method for generating a 3D model comprising:
continuously scanning a surface of a surgical site using a structured light scanner embodied on an endoscope;
continuously capturing images of a scan pattern projected on the surgical site by the structured light scanner;
generating a 3D model of the surgical site based on the captured images of the scan pattern at the based on the captured scan pattern;
detecting a change in the surgical site during the continuous scanning;
isolating a region of the surgical site where the change is detected;
flattening the isolated region in the 3D model;
capturing images of the scan pattern in just the isolation region of the surgical site; and
updating the 3D model generated using the captured images of the scan pattern in just the isolated regions of the surgical site, wherein when the detected change is a movement of a surgical tool through the scan pattern projected on the surgical site, the change is ignored and the continuously scanning and continuously capturing images continues.

### Claim 17 (depends on 16)

The method of claim 16, wherein detecting the change in the surgical site includes detecting movement of an object in the surgical site.

### Claim 18 (depends on 16)

The method of claim 16, wherein detecting the change in the surgical site includes:
comparing an image of the scan pattern captured a time t with an image of the scan pattern captured at a time t−1; and
determining that there exists the change in the surgical site when the image of the scan pattern captured at time t is different from the image of the scan pattern captured at time t−1.

### Claim 19 (depends on 16)

The method of claim 16 further comprising capturing optical images of the surgical site.

### Claim 20 (depends on 19)

The method of claim 19, wherein generating the 3D model of the surgical site includes warping the optical images onto the images of the scan pattern.

### Claim 21 (depends on 19)

The method of claim 19, wherein the optical images and the images of the scan pattern are simultaneously captured.
