# US 2025/0143543 A1 · Method for Real-Time Detection of Objects, Structures or Patterns in a Video, an Associated System and an Associated Computer Readable Medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0143543 A1
- Document ID: US20250143543A1
- Status: Published application
- Kind: A1
- Application number: 19/014,961
- Filed: 2025-01-09
- Published: 2025-05-08
- Earliest claimed: 2019-06-21
- Assignees: Augere Medical AS
- Applicants: Augere Medical AS
- Inventors: Håvard Nygaard Espeland; Michael Alexander Riegler
- Web page: https://patentgrep.com/patent/US20250143543A1

## CPC

- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- G06F 18/24133: {Distances to prototypes}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/56: relating to colour
- G06V 10/82: using neural networks
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 2201/031: of internal organs
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This invention relates to methods and systems for the real-time detection of objects, structures and/or patterns in videos, such as anatomical structures and/or anatomical landmarks in endoscopic videos of a subject, e.g. endoscopic videos of the gastrointestinal tract (GI tract).

## Claims

### Claim 1

A method for real-time detection of one or more objects and/or one or more structures and/or one or more patterns in a video, the method comprising
receiving a sequence of frames of the video;
applying a sliding window to the sequence of frames, and for each position of the sliding window, extracting one or more visual features from the frames within the sliding window, thereby generating a plurality of time images;
applying a trained classifier to each time image, wherein the trained classifier determines one or more detection scores that indicate likelihoods that a respective time image includes the one or more objects and/or one or more structures and/or one or more patterns; and
outputting, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns when a detection score of the one or more detection scores is higher than a detection threshold of the trained classifier.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the video is an endoscopic video.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the one or more objects and/or one or more structures and/or one or more patterns are one or more anatomical structures and/or one or more anatomical landmarks.

### Claim 4 (depends on 1)

The method of claim 1, wherein a size of the sliding window is dynamic.

### Claim 5 (depends on 1)

The method of claim 1, wherein the sliding window is overlapping.

### Claim 6 (depends on 1)

The method according to claim 1, wherein a sliding rate of the sliding window and a frame rate of the video are identical.

### Claim 7 (depends on 1)

The method according to claim 1, wherein extracting one or more visual features from the frames within the sliding window comprises extracting the one or more visual features using one or more algorithms for local feature extraction and/or global feature extraction and/or deep feature extraction.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the one or more visual features are deep features extracted through deep neural networks (DNN).

### Claim 9 (depends on 8)

The method according to claim 8, wherein the DNN is configured to implement one or more supervised training methods to extract such deep features.

### Claim 10 (depends on 1)

The method according to claim 1, wherein the trained classifier is trained for multi-class classification.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the trained classifier is a DNN or a capsule network adapted for analyzing images.

### Claim 12 (depends on 1)

The method according to claim 1, wherein outputting, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns comprises outputting a detection signal, preferably a visual alert or an audio alert, and/or data to a data storage unit.

### Claim 13 (depends on 1)

The method according to claim 1, wherein outputting, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns comprises outputting a visual alert to a display monitor.

### Claim 14 (depends on 13)

The method according to claim 13, wherein outputting, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns further comprises outputting an overlay video to a display monitor, and wherein the visual alert is overlaid over a video feed.

### Claim 15 (depends on 1)

The method according to claim 1, wherein prior to applying a sliding window to the sequence of frames, and for each position of the sliding window, extracting one or more visual features from the frames within the sliding window, thereby generating a plurality of time images, the method further comprises performing one or more pre-processing functions selected from the group consisting of noise removal, removal of black borders, cropping, resizing, blurring edges, and removal of metadata.

### Claim 16 (depends on 2)

The method according to claim 2, wherein the endoscopic video is of a gastrointestinal tract (GI tract).

### Claim 17 (depends on 16)

The method according to claim 16, wherein the endoscopic video is a colonoscopic video, and wherein the one or more anatomical structures are selected from the group consisting of healthy mucosa, stool, colonic fluid, blood vessels, inflamed mucosa, erosions, lesions, and polyps, and preferably wherein the one or more anatomical structures are polyps.

### Claim 18

A system for real-time detection of one or more objects and/or one or more structures and/or one or more patterns in a video, the system comprising:
an input configured to receive a sequence of frames of a video;
a processing system configured to access and process the sequence of frames, wherein to process the sequence of frames, the processing system is configured to:
receive a sequence of frames of the video;
apply a sliding window to the sequence of frames, and for each position of the sliding window, extract one or more visual features from the frames within the sliding window, thereby generating a plurality of time images; and
apply a trained classifier to each time image, wherein the trained classifier determines one or more detection scores that indicate a likelihood that a respective time image includes the one or more objects and/or one or more structures and/or one or more patterns; and
output, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns when a detection score of the one or more detection scores is higher than a detection threshold of the trained classifier.

### Claim 19

A non-transitory computer readable medium storing computer readable instructions for real-time detection of one or more objects and/or one or more structures and/or one or more patterns in a video, wherein the computer readable instructions, when executed by a processing system, cause the processing system to:
receive a sequence of frames of the video;
apply a sliding window to the sequence of frames, and for each position of the sliding window, extract one or more visual features from the frames within the sliding window, thereby generating a plurality of time images;
apply a trained classifier to each time image, wherein the trained classifier determines one or more detection scores that indicate a likelihood that a respective time image includes the one or more objects and/or one or more structures and/or one or more patterns; and
output, in real-time, the detection of the one or more objects and/or one or more structures and/or one or more patterns when a detection score of the one or more detection scores is higher than a detection threshold of the trained classifier.
