# US 2025/0157036 A1 · Multithreading video processing with cloud services to detect and analyze objects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157036 A1
- Document ID: US20250157036A1
- Status: Published application
- Kind: A1
- Application number: 19/027,716
- Filed: 2025-01-17
- Published: 2025-05-15
- Earliest claimed: 2022-04-12
- Assignees: Utech Products, Inc.
- Applicants: Utech Products, Inc.
- Inventors: Rakesh Madan; Zohair Hussain
- Web page: https://patentgrep.com/patent/US20250157036A1

## CPC

- G06T 1/20: Processor architectures; Processor configuration, e.g. pipelining
- G06T 7/0012: {Biomedical image inspection}
- A61B 1/000094: {extracting biological structures}
- A61B 1/000096: {using artificial intelligence}
- A61B 1/00016: {using wireless means}
- A61B 1/0002: {provided with data storages}
- A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled}
- A61B 1/31: for the rectum, e.g. proctoscopes or sigmoidoscopes
- G06V 20/41: {Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items (segmenting video sequences G06V20/49)}
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10068: Endoscopic image
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30028: Colon; Small intestine
- G06T 2207/30096: Tumor; Lesion
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

A system and method for processing a video stream captured during a clinical procedure. A method includes receiving image data from a scope during a procedure; forwarding the image data to a display as the scope is inserted into a patient; forwarding the image data to a cloud service to identify a landmark in the image data using a first machine learning model; receiving landmark details from the cloud service in response to a detected landmark and outputting the image data with landmark annotations; receiving and forwarding further image data to the display as the scope is further manipulated within the patient; forwarding the further image data to a cloud service that is configured to detect lesions using a second machine learning model; and receiving lesion details from the cloud service in response to a detected lesion and outputting the further image data with lesion annotations via the display.

## Claims

### Claim 1

A system, comprising:
a memory; and
a processor coupled to the memory and configured to process a video stream captured during a clinical procedure according to process that includes:
receiving and forwarding a current image frame from the video stream to a display manager using a first processing thread;
forwarding the current image frame to a memory buffer;
using a second processing thread that runs independently of the first processing thread to check for a new image frame in the memory buffer;
in response to the new image frame being detected in the memory buffer, using the second processing thread to retrieve and forward the new image frame to a cloud service that is configured to identify landmarks in image data;
receiving a result from the cloud service at the second processing thread;
in response to the result being indicative of a new landmark in the new image frame, generating landmark annotations; and
receiving, at the first processing thread, the landmark annotations and graphically overlaying the landmark annotations onto the current image frame at the display manager.

### Claim 2 (depends on 1)

The system of claim 1, wherein, prior to forwarding the current image frame to the memory buffer:
determining whether the second processing thread is busy; and
in response to determining that the second processing thread is busy, graphically overlaying previous landmark annotations onto the current image frame.

### Claim 3 (depends on 2)

The system of claim 2, wherein, in response to determining that the second processing thread is not busy, converting the current image frame to a JPEG format and loading current image frame into the memory buffer.

### Claim 4 (depends on 3)

The system of claim 3, wherein, prior to forwarding the new image frame to the cloud service, encoding the new image frame to a Base64 format.

### Claim 5 (depends on 1)

The system of claim 1, wherein in response to the result from the cloud service not being indicative of a new landmark in the new image frame, causing the first processing thread to graphically overlay previous landmark annotations onto the current image frame.

### Claim 6 (depends on 5)

The system of claim 5, wherein the previous landmark annotations are obtained from the memory buffer.

### Claim 7 (depends on 1)

The system of claim 1, wherein forwarding new landmark annotations to the first processing thread includes:
loading the new landmark annotations to the memory buffer by the second processing thread; and
using the first processing thread to detect new landmark annotations in the memory buffer.

### Claim 8 (depends on 1)

The system of claim 1, wherein the first processing thread and second processing run on separate central processing unit (CPU) cores, which are separate from the cloud service.

### Claim 9

A system, comprising:
a memory; and
a processor coupled to the memory and configured to process image data streamed from a scope during a clinical procedure according to process that includes:
receiving and forwarding image data to a display as the scope is inserted into a patient;
forwarding the image data to a cloud service that is configured to identify a landmark in the image data using a first machine learning model;
receiving landmark details from the cloud service in response to a detected landmark and outputting the image data with landmark annotations via the display;
receiving and forwarding further image data to the display as the scope is further manipulated within the patient;
forwarding the further image data to a cloud service that is configured to detect lesions using a second machine learning model; and
receiving lesion details from the cloud service in response to a detected lesion and outputting the further image data with lesion annotations via the display.

### Claim 10 (depends on 9)

The system of claim 9, wherein further manipulation of the scope within the patient comprises a retraction of the scope.

### Claim 11 (depends on 9)

The system of claim 9, wherein outputting of the image data with landmark annotations and further image data with lesion annotations occurs while the scope is within the patient.

### Claim 12 (depends on 9)

The system of claim 9, wherein:
receiving and forwarding image data and further image data to the display is implemented with a first processing thread; and
forwarding the image data and further image data to a cloud service is implemented with a second processing thread.

### Claim 13 (depends on 12)

The system of claim 12, wherein the first processing thread runs on a first central processing unit (CPU) core and the second processing thread runs on a second CPU core, and wherein the first and second CPU cores are independent of the cloud service.

### Claim 14 (depends on 9)

The system of claim 9, wherein the procedure comprises a gastrointestinal procedure and the landmark comprises a cecum.

### Claim 15 (depends on 9)

The system of claim 9, wherein in response to the detected lesion, utilizing a third machine learning model to size the lesion.

### Claim 16 (depends on 15)

The system of claim 15, wherein in response to the detected lesion, utilizing a fourth machine learning model to diagnose the lesion.

### Claim 17

A method, comprising:
receiving image data streamed from a scope during a clinical procedure;
forwarding the image data to a display as the scope is inserted into a patient;
forwarding the image data to a cloud service that is configured to identify a landmark in the image data using a first machine learning model;
receiving landmark details from the cloud service in response to a detected landmark and outputting the image data with landmark annotations via the display;
receiving and forwarding further image data to the display as the scope is further manipulated within the patient;
forwarding the further image data to a cloud service that is configured to detect lesions using a second machine learning model; and
receiving lesion details from the cloud service in response to a detected lesion and outputting the further image data with lesion annotations via the display.

### Claim 18 (depends on 17)

The method of claim 17, wherein the procedure comprises a gastrointestinal procedure and the landmark comprises a cecum, and wherein further manipulation of the scope within the patient comprises a retraction of the scope.

### Claim 19 (depends on 17)

The method of claim 17, wherein:
receiving and forwarding image data and further image data to the display is implemented with a first processing thread; and
forwarding the image data and further image data to a cloud service is implemented with a second processing thread.

### Claim 20 (depends on 19)

The method of claim 19, wherein the first processing thread runs on a first central processing unit (CPU) core and the second processing thread runs on a second CPU core, and wherein the first and second CPU cores are independent of the cloud service.
