# US 11,308,614 B2 · Deep learning for real-time colon polyp detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,308,614 B2
- Document ID: US11308614B2
- Status: Granted
- Kind: B2
- Application number: 17/020,240
- Filed: 2020-09-14
- Published: 2021-05-06
- Granted: 2022-04-19
- Earliest claimed: 2018-03-20
- Applicants: EndoVigilant Inc.
- Inventors: Vineet Sachdev; Ravindra Kompella; Hamed Pirsiavash
- Web page: https://patentgrep.com/patent/US11308614B2

## CPC

- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 5/80: Geometric correction
- G06T 5/94: based on local image properties, e.g. for local contrast enhancement
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/593: from stereo images
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/10068: Endoscopic image
- G06T 2207/20036: Morphological image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20152: Watershed segmentation
- G06T 2207/30028: Colon; Small intestine
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A set of enhancements to further improve the performance of deep learning artificial intelligence algorithms trained to detect and localize colon polyps. The enhancements spanning training data mining efficiencies and automation, training data augmentation, early detection of polyps enable a more performant colon polyp detection solution for use on colonoscopy procedure recordings or live procedures in endoscopy centers.

## Claims

### Claim 1

A method of enhancing training of a colon polyp detector using a deep learning-based object detector, comprising:
gathering a polyp early appearance image training dataset that includes a plurality of video clips each including an initial image of a polyp that is just starting to appear in frame;
sampling the plurality of video clips in the initial portion including the initial image of the polyp at a first sampling rate to generate polyp early appearance image training data;
sampling a remainder portion of the plurality of video clips at a second sampling rate, the second sampling rate being slower than the first sampling rate, to generate polyp image training data; and
training the colon polyp detector with both the polyp early appearance image training data and with the polyp image training data.

### Claim 2 (depends on 1)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 1, further comprising:
sampling the remainder portion of each of the plurality of video clips after the initial region at a lower image sampling rate than for the initial region.

### Claim 3 (depends on 1)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 1, wherein
the initial portion has a predetermined fixed length of time.

### Claim 4 (depends on 1)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 1, wherein:
the colon polyp detector training is further enhanced by additionally training the colon polyp detector with a complementary transformed image dataset including a transformation of the polyp early appearance image training data, in addition to training with the polyp early appearance image training data and the polyp image training data;
whereby accuracy of polyp detection is increased at time of inference by the addition of transformed images in the training.

### Claim 5 (depends on 4)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 4, wherein:
the transformation of the polyp early appearance image training data is at least one of shifting and rotating an image within the polyp early appearance image training data.

### Claim 6 (depends on 4)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 4, wherein:
the transformation of the polyp early appearance image training data is generated by shifting an image within the polyp image training data.

### Claim 7 (depends on 4)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 4, wherein:
the transformation of the polyp early appearance image training data is generated by rotating an image within the polyp image training data.

### Claim 8 (depends on 7)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 7, wherein:
the transformation rotates the image at a random angle.

### Claim 9 (depends on 4)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 4, wherein:
the transformation of the polyp early appearance image training data is generated by altering at least one of hue, saturation, and exposure within a limited range.

### Claim 10 (depends on 1)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 1, further comprising:
training the colon polyp detector to minimize a difference in time in the plurality of video clips between when the polyp first appears in frame, and when the colon polyp detector first detects that polyp.

### Claim 11 (depends on 1)

The method of enhancing training of a colon polyp detector using a deep learning-based object detector according to claim 1, further comprising:
further training the colon polyp detector with images of missed polyp detection.