# US 11,445,108 B1 · Turn direction guidance of an endoscopic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,445,108 B1 - Document ID: US11445108B1 - Status: Granted - Kind: B1 - Application number: 17/249,557 - Filed: 2021-03-05 - Published: 2022-09-08 - Granted: 2022-09-13 - Earliest claimed: 2021-03-05 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Saniya Ben Hassen; Marc P. Yvon; Anthony Herve; Christel Beaujard - Web page: https://patentgrep.com/patent/US11445108B1 ## CPC - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - A61B 1/00006: {of control signals} - A61B 1/00009: {of image signals during a use of endoscope} - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - G06F 18/22: Matching criteria, e.g. proximity measures - G06T 7/0012: {Biomedical image inspection} - G06T 7/60: Analysis of geometric attributes - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/73: using feature-based methods - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components - H04N 7/183: {for receiving images from a single remote source} - H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - A61B 1/0676: {at distal tip of an endoscope} - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2090/061: {for measuring dimensions, e.g. length} - G06T 2207/10068: Endoscopic image - G06T 2207/30004: Biomedical image processing - G06T 2207/30028: Colon; Small intestine - G06V 2201/031: of internal organs - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Method and system are provided for direction guidance of an endoscopic device in a tubular organ. The method receives a current image frame from a camera disposed on the endoscopic device, where the current image frame captures a visible lumen of the tubular organ. The method determines an area of the visible lumen in the current image frame and a ratio of the area of the visible lumen to a minimum enclosing circle of the visible lumen. If the ratio breaches a defined threshold, the method adjusts a target direction of the endoscopic device in a direction from a center of the visible lumen towards a center of the image frame and outputs a notification of the adjusted target direction to a controller of the endoscopic device. ## Claims ### Claim 1 A computer-implemented method for turn direction guidance of an endoscopic device in real-time in a tubular organ, comprising: receiving a current image frame from a camera disposed on the endoscopic device, wherein the current image frame captures a visible lumen of the tubular organ; determining an area of the visible lumen in the current image frame; determining a ratio of an area of a minimum enclosing circle of the visible lumen to the area of the visible lumen; upon determining the ratio breaches a defined threshold, adjusting a target direction of the endoscopic device in a direction from a center of the visible lumen towards a center of the current image frame; outputting a notification of the adjusted target direction to a controller of the endoscopic device; comparing the ratio of the current image frame with a ratio of a previous image frame; determining there is a decrease in a size of the area of the visible lumen indicating an approaching turn in the tubular organ; and upon determining there is a decrease in the size of the area of the visible lumen, outputting a notification of the approaching turn to the controller of the endoscopic device. ### Claim 2 (depends on 1) The computer-implemented method according to claim 1, wherein upon determining the ratio breaches the defined threshold, adjusting the target direction by a distance proportional to the ratio from an existing direction towards the center of the image frame. ### Claim 3 (depends on 2) The computer-implemented method according to claim 2, wherein the distance is a measurement between the center of the visible lumen and the center of the image frame. ### Claim 4 (depends on 1) The computer-implemented method according to claim 1, wherein the determining the the ratio breaches the defined threshold is determined when the visible lumen is a configured amount less than the area of the minimum enclosing circle, indicating that turn is in progress. ### Claim 5 (depends on 1) The computer-implemented method according to claim 1, wherein the controller of the endoscopic device is an automated controller that automatically adjusts the target direction of the endoscopic device. ### Claim 6 (depends on 1) The computer-implemented method according to claim 1, further comprising: outputting a notification of the adjusted target direction to the controller of the endoscopic device; and providing a visual indication of a target direction on a user interface. ### Claim 7 (depends on 1) The computer-implemented method according to claim 1, further comprising: including iterating the method for each of a selection of image frames received from the camera and outputting notifications in real-time. ### Claim 8 (depends on 1) The computer-implemented method according to claim 1, wherein the visible lumen is determined by identifying the darkest pixels in the image frame, and a center of the visible lumen is calculated as a barycenter of the area of the visible lumen. ### Claim 9 A computer program product for turn direction guidance of an endoscopic device in a tubular organ, the computer program product comprising: one or more computer-readable tangible storage medium and program instructions stored on at least one of the one or more tangible storage medium, the program instructions executable by a processor, the program instructions the program instructions readable by a computing system to cause the computing system to perform a method comprising: receiving a current image frame from a camera disposed on the endoscopic device, wherein the current image frame captures a visible lumen of the tubular organ; determining an area of the visible lumen in the current image frame; determining a ratio of a minimum enclosing circle of the visible lumen to the area of the visible lumen; upon determining the ratio breaches a defined threshold, adjusting a target direction of the endoscopic device from a center of the visible lumen towards a center of the current image frame; outputting a notification of the adjusted target direction to a controller of the endoscopic device; comparing the ratio of the current image frame with a ratio of a previous image frame; determining there is a decrease in a size of the area of the visible lumen indicating an approaching turn in the tubular organ; upon determining there is a decrease in the size of the area of the visible lumen, outputting a notification of the approaching turn to the controller of the endoscopic device. ### Claim 10 A computer system for turn direction guidance of an endoscopic device in real-time in a tubular organ, the computer system comprising: one or more computer processors, one or more computer-readable storage media, and program instructions stored on the one or more of the computer-readable storage media for execution by at least one of the one or more processors, wherein the computer system is capable of performing a method comprising: receiving a current image frame from a camera disposed on the endoscopic device, wherein the current image frame captures a visible lumen of the tubular organ; determining an area of the visible lumen in the current image frame; determining a ratio of an area of a minimum enclosing circle of the visible lumen to the area of the visible lumen; upon determining the ratio breaches a defined threshold, adjusting a target direction of the endoscopic device in a direction from a center of the visible lumen towards a center of the current image frame; outputting a notification of the adjusted target direction to a controller of the endoscopic device comparing the ratio of the current image frame with a ratio of a previous image frame; determining there is a decrease in a size of the area of the visible lumen indicating an approaching turn in the tubular organ; and upon determining there is a decrease in the size of the area of the visible lumen, outputting a notification of the approaching turn to the controller of the endoscopic device. ### Claim 11 (depends on 10) The computer system according to claim 10, wherein upon determining the ratio breaches the defined threshold, adjusting the target direction by a distance proportional to the ratio from an existing direction towards the center of the image frame. ### Claim 12 (depends on 11) The computer system according to claim 11, wherein the distance is a measurement between the center of the visible lumen and the center of the image frame. ### Claim 13 (depends on 10) The computer system according to claim 10, wherein the determining the ratio breaches the defined ratio is determined when the visible lumen is a configured amount less than the area of the minimum enclosing circle, indicating that turn is in progress. ### Claim 14 (depends on 10) The computer system according to claim 10, wherein the controller of the endoscopic device is an automated controller that automatically adjusts the target direction of the endoscopic device. ### Claim 15 (depends on 10) The computer system according to claim 10, further comprising: outputting a notification of the adjusted target direction to the controller of the endoscopic device; and providing a visual indication of a target direction on a user interface. ### Claim 16 (depends on 10) The computer system according to claim 10, further comprising: including iterating the method for each of a selection of image frames received from the camera and outputting notifications in real-time. ### Claim 17 (depends on 10) The computer system according to claim 10, wherein the visible lumen is determined by identifying the darkest pixels in the image frame, and a center of the visible lumen is calculated as a barycenter of the area of the visible lumen. ### Claim 18 A computer-implemented method for turn direction guidance of an endoscopic device in real-time in a tubular organ, comprising: receiving a current image frame from a camera disposed on the endoscopic device, wherein the current image frame captures a visible lumen of the tubular organ; determining an area of the visible lumen in the current image frame; determining a ratio of an area of a minimum enclosing circle of the visible lumen to the area of the visible lumen; upon determining the ratio breaches a defined threshold, adjusting a target direction of the endoscopic device in a direction from a center of the visible lumen towards a center of the current image frame; outputting a notification of the adjusted target direction to a controller of the endoscopic device; comparing the ratio of the current image frame with a ratio of a previous image frame; determining there is an increase in a size of the area of the visible lumen indicating a moving away from a turn in the tubular organ; and upon determining there is an increase in the size of the area of the visible lumen, outputting a notification of a the moving away from the turn to the controller of the endoscopic device.