# US 11,304,597 B2 · Endoscope apparatus and operating method of endoscope apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,304,597 B2 - Document ID: US11304597B2 - Status: Granted - Kind: B2 - Application number: 16/843,013 - Filed: 2020-04-08 - Published: 2020-07-23 - Granted: 2022-04-19 - Earliest claimed: 2017-10-24 - Assignees: Olympus Corporation - Applicants: Olympus Corporation - Inventors: Koichiro Yoshino; Naoya Kuriyama - Web page: https://patentgrep.com/patent/US11304597B2 ## CPC - A61B 1/00188: {with focusing or zooming features} - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/045: Control thereof - A61B 1/0655: {Control therefor} - G02B 7/38: measured at different points on the optical axis {, e.g. focussing on two or more planes and comparing image data} - G02B 23/2484: {Arrangements in relation to a camera or imaging device (processing or control of video signals generated by an endoscope H04N23/60, H04N23/70)} - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - H04N 23/56: provided with illuminating means - H04N 23/67: Focus control based on electronic image sensor signals - H04N 23/676: {Bracketing for image capture at varying focusing conditions} - H04N 23/73: by influencing the exposure time - H04N 23/74: by influencing the scene brightness using illuminating means - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 23/959: by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics - A61B 1/00045: {Display arrangement} - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - A61B 1/0669: {at proximal end of an endoscope} - A61B 1/0684: {using light emitting diodes [LED]} - A61B 1/07: using light-conductive means, e.g. optical fibres - G02B 23/243: {Objectives for endoscopes} - H04N 25/531: by controlling rolling shutters in CMOS SSIS - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An endoscope apparatus includes a processor. The processor performs controlling a focus position of an objective optical system, acquiring images sequentially captured by an image sensor, and combining the images in N frames thus captured into a depth of field extended image in one frame. The processor controls the focus position such that focus positions at timings when the respective images in N frames are captured differ from each other. The processor combines the images in N frames that have been controlled to receive a constant quantity of light emission of illumination light or the images in N frames that have undergone a correction process to make image brightness constant, into the depth of field extended image. ## Claims ### Claim 1 An endoscope apparatus comprising: a processor configured to: control a focus position of an objective optical system, the objective optical system being configured to form an image of reflected light from a subject on an image sensor, the subject reflecting illumination light emitted thereon, such that focus positions at timings when respective images in N frames are sequentially captured by the image sensor differ from each other, N being an integer of two or more; control a quantity of light emission of the illumination light such that quantities of light emission of the illumination light when the respective images in N frames are captured become equal to each other, or performing a correction process of at least one of the respective images such that brightness of the respective images in N frames becomes equal to each other; and after controlling the focus position and controlling the quantities of light emission or performing the correction process, combine the respective images in N frames, each having different focused parts, into a depth of field extended image in one frame by coupling the focused parts with each other. ### Claim 2 (depends on 1) The endoscope apparatus as defined in claim 1, wherein the processor is configured to control the illumination light to adjust brightness of the depth of field extended image, and maintain the quantity of light emission to be constant when the respective images in N frames are captured. ### Claim 3 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to control emission of the illumination light in a light emission period including a stop timing of the focus position. ### Claim 4 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to: control the focus position to reciprocate the focus position between a first focus position and a second focus position that differs from the first focus position; control emission of the illumination light in a first light emission period including a timing when the focus position is at the first focus position and in a second light emission period including a timing when the focus position is at the second focus position; and combine the respective images in N frames including an image captured in the first light emission period and an image captured in the second light emission period into the depth of field extended image. ### Claim 5 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the image sensor is configured to capture the respective images in N frames by a rolling shutter system, and wherein the processor is configured to control emission of the illumination light in a whole-pixel exposure period when all lines in an effective pixel region of the image sensor are in an exposure state. ### Claim 6 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to control a length of a light emission period of the illumination light to control a total quantity of light emission when each of the respective images in N frames is captured. ### Claim 7 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to control a quantity of light emission of the illumination light per unit time to control a total quantity of light emission when each of the respective images in N frames is captured. ### Claim 8 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to control emission of the illumination light in a light emission period when a center of an exposure period of the image sensor coincides with a center of the light emission period. ### Claim 9 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to: control emission of the illumination light by pulse light emission a plurality of times in an exposure period of the image sensor, wherein a gravity center of a total quantity of light emission of the pulse light emission coincides with a stop timing of the focus position, and wherein the gravity center is a timing when a total quantity of light emission of the pulse light emission before the stop timing of the focus position coincides with a total quantity of light emission of the pulse light emission after the stop timing of the focus position. ### Claim 10 (depends on 2) The endoscope apparatus as defined in claim 2, wherein the processor is configured to: combine images in a first set of N frames into a first depth of field extended image and images in a second set of N frames captured after the images in the first set of N frames into a second depth of field extended image; and change the quantity of light emission of the illumination light in a period between a period when the images in the first set of N frames are captured and a period when the images in the second set of N frames are captured. ### Claim 11 (depends on 1) The endoscope apparatus as defined in claim 1, wherein the processor is configured to: control emission of the illumination light to adjust brightness of the respective images in N frames captured by the image sensor; and perform the correction process to make the brightness of the respective images in N frames constant, and combine the images in N frames that have undergone the correction process into the depth of field extended image. ### Claim 12 (depends on 1) The endoscope apparatus as defined in claim 1, comprising: an illumination device configured to emit the illumination light to the subject; and an imaging device comprising the objective optical system and the image sensor. ### Claim 13 An operating method of an endoscope apparatus comprising: controlling a focus position of an objective optical system, the objective optical system being configured to form an image of reflected light from a subject on an image sensor, the subject reflecting illumination light emitted thereon, such that focus positions at timings when respective images in N frames are sequentially captured by the image sensor differ from each other, N being an integer of two or more; controlling a quantity of light emission of the illumination light such that quantities of light emission of the illumination light when the respective images in N frames are captured become equal to each other, or performing a correction process of at least one of the respective images such that brightness of the respective images in N frames becomes equal to each other; and after controlling the focus position and controlling the quantities of light emission or performing the correction process, combining the respective images in N frames, each having different focused parts, into a depth of field extended image in one frame by coupling the focused parts with each other. ### Claim 14 An endoscope system comprising: an endoscope comprising an image sensor; and an endoscope apparatus comprising: a processor configured to: control a focus position of an objective optical system; acquire images sequentially captured by the image sensor; combine the images in N frames (N is an integer of two or more) each having different focused parts thus acquired into a depth of field extended image in one frame, by coupling the focused parts with each other; and control the illumination light before combination into the depth of field extended image in one frame; control the focus position such that focus positions at timings when the respective images in N frames are captured differ from each other; combine images in a first set of N frames into a first depth of field extended image and images in a second set of N frames captured after the images in the first set of N frames into a second depth of field extended image; and change a quantity of light emission of the illumination light in a period between a period when the images in the first set of N frames are captured and a period when the images in the second set of N frames are captured, or perform, before combination into the depth of field extended image in one frame, a correcting process of at least one of the respective images to equalize brightness of the respective images in N frames. ### Claim 15 (depends on 1) The endoscope apparatus as defined in claim 1, wherein the image sensor is configured to acquire the images at a first focus position and a second focus position that differs from the first focus position. ### Claim 16 (depends on 15) The endoscope apparatus as defined in claim 15, wherein the first focus position is on a far point side of the second focus position.