# US 11,432,707 B2 · Endoscope system, processor for endoscope and operation method for endoscope system for determining an erroneous estimation portion > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,432,707 B2 - Document ID: US11432707B2 - Status: Granted - Kind: B2 - Application number: 16/713,319 - Filed: 2019-12-13 - Published: 2020-05-21 - Granted: 2022-09-06 - Earliest claimed: 2017-06-15 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Shunya Akimoto; Soichi Ikuma - Web page: https://patentgrep.com/patent/US11432707B2 ## CPC - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/00045: {Display arrangement} - A61B 1/00193: {adapted for stereoscopic vision} - A61B 1/00194: {adapted for three-dimensional imaging} - A61B 1/0669: {at proximal end of an endoscope} - A61B 5/065: {Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe} - A61B 5/1076: {for measuring dimensions inside body cavities, e.g. using catheters (A61B3/1005 takes precedence)} - G02B 23/24: Instruments {or systems} for viewing the inside of hollow bodies, e.g. fibrescopes - G06T 7/0012: {Biomedical image inspection} - G06T 7/50: Depth or shape recovery - G06T 7/593: from stereo images - G06T 17/30: Polynomial surface description - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - A61B 2090/309: {using white LEDs} - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/10068: Endoscopic image - G06T 2207/30084: Kidney; Renal - G06T 2207/30241: Trajectory - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An endoscope system includes an endoscope that acquires an image of a subject and a processor. The processor acquires position and posture information of the endoscope, estimates a three-dimensional shape of the subject based on the image and the position and posture information, generates, based on the position information, a cross section crossing a trajectory on which the endoscope passed, and determines an erroneous estimation portion based on a shape characteristic of the three-dimensional shape on the cross section. ## Claims ### Claim 1 An endoscope system comprising: an endoscope that picks up an image in a subject and acquires the image; and a processor, wherein the processor acquires position information and posture information of the endoscope, estimates, based on the image, a relative three-dimensional position viewed from the endoscope of a target portion in the subject, estimates a three-dimensional shape in the subject based on the relative three-dimensional position and the position information and the posture information, generates a cross section, which is a plane crossing a centerline of a region including a trajectory on which the endoscope passed, based on the position information, and determines an erroneous estimation portion in the three-dimensional shape based on a shape characteristic of a shape cross section, which is a portion crossing the cross section, of the three-dimensional shape. ### Claim 2 (depends on 1) The endoscope system according to claim 1, wherein the processor estimates the three-dimensional shape as a shape obtained by connecting a plurality of polygons, the shape cross section is formed in a polygonal line obtained by connecting a plurality of intersection line segments, which are intersection lines of the cross section and the polygons, and the processor determines, based on each of the intersection line segments, whether the polygon including the intersection line segment is the erroneous estimation portion. ### Claim 3 (depends on 2) The endoscope system according to claim 2, wherein the processor generates, with respect to each of the plurality of intersection line segments, a connecting line segment having, as both ends, a trajectory point that is an intersection of the cross section and the trajectory, and a center point of the intersection line segment and determines, as the erroneous estimation portion, the polygon including the intersection line segment crossing the connecting line segment in a part other than both the ends of the connecting line segment. ### Claim 4 (depends on 3) The endoscope system according to claim 3, wherein, when the trajectory point in plurality is present on one cross section, the processor generates the connecting line segment with one end set at a trajectory center of gravity point that is a center of gravity of the plurality of trajectory points, and determines the erroneous estimation portion. ### Claim 5 (depends on 3) The endoscope system according to claim 3, wherein, when the trajectory point in plurality is present on one cross section, the processor generates the connecting line segment with respect to each of the plurality of trajectory points and determines the erroneous estimation portion. ### Claim 6 (depends on 2) The endoscope system according to claim 2, wherein the processor generates, with respect to each of the plurality of intersection line segments, a center vector from a trajectory center of gravity point to a center point of the intersection line segment, the trajectory center of gravity point being a center of gravity of a trajectory point that is an intersection of the cross section and the trajectory, generates, with respect to each of the polygons including the intersection line segments, projected normal vectors obtained by projecting normal vectors of the polygons onto the cross section, and determines, as the erroneous estimation portion, the polygon including the intersection line segment in which an angle formed by the center vector and the projected normal vector is equal to or larger than an angle threshold. ### Claim 7 (depends on 2) The endoscope system according to claim 2, wherein the processor generates, with respect to each of the polygons including the intersection line segments, projected normal vectors obtained by projecting normal vectors of the polygons onto the cross section, and determines, as the erroneous estimation portion, a pair of the polygons in which an angle formed by the projected normal vectors is equal to or larger than an angle threshold, a linear distance between center points of the intersection line segments is smaller than a first threshold, and a distance along the shape cross section between the center points of the intersection line segments is equal to or larger than a second threshold that is larger than the first threshold. ### Claim 8 (depends on 2) The endoscope system according to claim 2, wherein the processor calculates a center gravity center point, the center gravity center point being a center of gravity of center points of all the intersection line segments configuring the shape cross section, generates, with respect to each of the plurality of intersection line segments, straight lines passing the center gravity center point and the center points of the intersection line segments, and determines, as the erroneous estimation portion, a polygon closer to the centerline of a pair of the polygons crossing the straight lines, a linear distance between two points where the straight lines cross the shape cross section being equal to or smaller than a threshold. ### Claim 9 (depends on 1) The endoscope system according to claim 1, wherein the processor generates, along the centerline, at every constant interval, a plurality of cross sections perpendicular to the centerline. ### Claim 10 (depends on 1) The endoscope system according to claim 1, wherein the processor deletes the erroneous estimation portion from the three-dimensional shape. ### Claim 11 (depends on 1) The endoscope system according to claim 1, wherein the processor displays the erroneous estimation portion in a display form different from a display form of other portions in the three-dimensional shape. ### Claim 12 A processor for an endoscope that acquires position information and posture information of an endoscope, estimates, based on an image acquired by the endoscope, a relative three-dimensional position viewed from the endoscope of a target portion in a subject, estimates a three-dimensional shape in the subject based on the relative three-dimensional position and the position information and the posture information, generates a cross section, which is a plane crossing a centerline of a region including a trajectory on which the endoscope passed, based on the position information, and determines an erroneous estimation portion in the three-dimensional shape based on a shape characteristic of a shape cross section, which is a portion crossing the cross section, of the three-dimensional shape. ### Claim 13 An operation method for an endoscope system, the endoscope system including an endoscope that picks up an image in a subject and acquires the image, the operation method comprising: acquiring position information and posture information of the endoscope; estimating, based on the image, a relative three-dimensional position viewed from the endoscope of a target portion in the subject; estimating a three-dimensional shape in the subject based on the relative three-dimensional position and the position information and the posture information; generating a cross section, which is a plane crossing a centerline of a region including a trajectory on which the endoscope passed, based on the position information; and determining an erroneous estimation portion in the three-dimensional shape based on a shape characteristic of a shape cross section, which is a portion crossing the cross section, of the three-dimensional shape.