# US 10,869,595 B2 · Endoscope system, controller, and computer-readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,869,595 B2 - Document ID: US10869595B2 - Status: Granted - Kind: B2 - Application number: 15/974,989 - Filed: 2018-05-09 - Published: 2018-09-13 - Granted: 2020-12-22 - Earliest claimed: 2015-11-13 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Shintaro Inoue - Web page: https://patentgrep.com/patent/US10869595B2 ## CPC - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/00: Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes (examination of body cavities or body tracts using ultrasonic, sonic or infrasonic waves A61B8/12; endoscopic instruments for taking cell samples or for biopsy A61B10/04); Illuminating arrangements therefor (for the eyes A61B3/00) - A61B 1/00006: {of control signals} - A61B 1/045: Control thereof - A61B 1/053: {being detachable} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/73: using feature-based methods - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - G06T 2207/10016: Video; Image sequence - G06T 2207/10068: Endoscopic image - G06T 2207/30244: Camera pose - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The technology disclosed herein is directed to an endoscope system. The endoscope system includes an endoscope, a sensor; and a controller. The controller is configured to control the endoscope. The controller includes one or more processors. The one or more processors is configured (i) to extract first feature points from a first image captured by the endoscope, (ii) to extract second feature points from a second image captured by the endoscope after acquisition of the first image, (iii) to estimate a first result of position and orientation of the endoscope based on the first feature points and the second feature points, (iv) to calculate a first estimation accuracy based on the first result. When the first estimation accuracy is greater than a predetermined value, the one or more processors utilizes the first result as position and orientation of the endoscope. ## Claims ### Claim 1 An endoscope system comprising: an endoscope; a sensor configured to detect at least one of position, orientation, movement speed, and acceleration of the endoscope; and a controller configured to control the endoscope, the controller comprises one or more processors, wherein the one or more processors being configured to: extract first feature points from a first image captured by the endoscope, extract second feature points from a second image captured by the endoscope after acquisition of the first image, estimate a first result of position and orientation of the endoscope based on the first feature points and the second feature points, and calculate a first estimation accuracy based on the first result of position and orientation of the endoscope, and wherein when the first estimation accuracy is greater than a predetermined value, the one or more processors is configured to utilize the first result as position and orientation of the endoscope. ### Claim 2 (depends on 1) The endoscope system according to claim 1, wherein the sensor is attached to at least one of the endoscope and a trocar. ### Claim 3 (depends on 1) The endoscope system according to claim 1, wherein when the first estimation accuracy is equal to or smaller than the predetermined value, the one or more processors being further configured to: receive a first value detected by the sensor at the time of capturing the first image, and receive a second value detected by the sensor at the time of capturing the second image, estimate a second result of position and orientation of the endoscope based on the first value and the second value, and calculate a second estimation accuracy based on the second result, so that when the second estimation accuracy is greater than the predetermined value, the one or more processors is configured to utilize the second result as position and orientation of the endoscope. ### Claim 4 (depends on 3) The endoscope system according to claim 3, wherein the one or more processors being further configured to estimate a position of an insertion point of the endoscope into a body of the patient based on the first result or the second result. ### Claim 5 (depends on 1) The endoscope system according to claim 1, wherein the one or more processors being configured to: extract coordinates of corresponding points by associating the first feature points with the second feature points; and estimate the first result based on the coordinates of the corresponding points. ### Claim 6 A controller for controlling an endoscope system including an endoscope, the controller comprising: one or more processors being configured to: extract first feature points from a first image captured by the endoscope; extract second feature points from a second image captured by the endoscope after acquisition of the first image; estimate a first result of position and orientation of the endoscope based on the first feature points and the second feature points; and calculate a first estimation accuracy based on the first result of position and orientation of the endoscope, wherein when the first estimation accuracy is greater than a predetermined value, the one or more processors is configured to utilize the first result as position and orientation of the endoscope, wherein, when the first estimation accuracy is equal to or smaller than the predetermined value, the one or more processors being further configured to: receive a first value detected by a sensor at the time of capturing the first image; receive a second value detected by the sensor at the time of capturing the second image; estimate a second result of position and orientation of the endoscope based on the first value and the second value; and calculate a second estimation accuracy based on the second result, so that when the second estimation accuracy is greater than the predetermined value: the one or more processors is configured to utilize the second result as position and orientation of the endoscope, and wherein the sensor is configured to detect at least one of position, orientation, movement speed, and acceleration of the endoscope. ### Claim 7 (depends on 6) The controller according to claim 6, wherein the one or more processors being further configured to estimate a position of an insertion point of the endoscope into a body of patient based on the first result or the second result. ### Claim 8 (depends on 6) The controller according to claim 6, wherein the one or more processors being configured to: extract coordinates of corresponding points by associating the first feature points with the second feature points; and estimate the first result based on the coordinates of the corresponding points. ### Claim 9 (depends on 6) The controller according to claim 6, wherein the sensor is attached to at least one of the endoscope and a trocar. ### Claim 10 A non-transitory computer-readable medium storing a computer-readable program for implementing controlling a method with an endoscope system including an endoscope, the method comprising: extracting first feature points from a first image captured by the endoscope; extracting second feature points from a second image captured by the endoscope after acquisition of the first image; estimating a first result of position and orientation of the endoscope based on the first feature points and the second feature points; and calculating a first estimation accuracy based on the first result of position and orientation of the endoscope so that when the first estimation accuracy is greater than a predetermined value, utilizing the first result as position and orientation of the endoscope, wherein, when the first estimation accuracy is equal to or smaller than the predetermined value, the method further comprising: receiving a first value detected by a sensor at the time of capturing the first image; receiving a second value detected by the sensor at the time of capturing the second image; estimating a second result of position and orientation of the endoscope based on the first value and the second value; and calculating a second estimation accuracy based on the second result, so that when the second estimation accuracy is greater than the predetermined value, utilizing the second result as position and orientation of the endoscope, and wherein the sensor is configured to detect at least one of position, orientation, movement speed, and acceleration of the endoscope. ### Claim 11 (depends on 10) The non-transitory computer-readable medium according to claim 10, wherein the method further comprising estimating a position of an insertion point of the endoscope into a body of patient based on the first result or the second result. ### Claim 12 (depends on 10) The non-transitory computer-readable medium according to claim 10, wherein the method comprising: extracting coordinates of corresponding points by associating the first feature points with the second feature points; and estimating the first result based on the coordinates of the corresponding points. ### Claim 13 (depends on 10) The non-transitory computer-readable medium according to claim 10, wherein the sensor is attached to at least one of the endoscope and a trocar.