# US 12,599,294 B2 · Image-recording device for improved low light intensity imaging and associated image-recording method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,599,294 B2 - Document ID: US12599294B2 - Status: Granted - Kind: B2 - Application number: 17/862,674 - Filed: 2022-07-12 - Published: 2023-01-19 - Granted: 2026-04-14 - Earliest claimed: 2021-07-16 - Assignees: Schölly Fiberoptic GmbH - Applicants: SchöLly Fiberoptic GmbH - Inventors: Michael Schwärzle; Matthias Kühn; Johannes Bourbon; Stefan Schröer; Lutz Labusch - Web page: https://patentgrep.com/patent/US12599294B2 ## CPC - A61B 1/05: characterised by the image sensor, e.g. camera, being in the distal end portion - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/043: {for fluorescence imaging} - A61B 1/0638: {providing two or more wavelengths} - H04N 13/128: Adjusting depth or disparity - H04N 13/156: Mixing image signals - H04N 13/239: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N 23/16: Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction - H10F 39/182: {Colour image sensors} - H10F 39/8053: {Colour filters} - A61B 1/00186: {with imaging filters} - A61B 2562/046: in a matrix array - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION ## Abstract In order to improve imaging, in particular for low light intensities, an image recording device, in particular formed as an endoscope, is proposed, which comprises a single photon sensitive detector (SPSD) in addition to an image sensor, which uses photodiodes as light-sensitive cells, in order to respectively detect light from a common object area. With the help of the SPSD, additional image information can be obtained from the object area, to improve the image data recorded with the image sensor or to enhance it with additional image information, in particular with regard to a further spectral range, which is captured with the SPSD. ## Claims ### Claim 1 An image recording device (1) arranged for medical imaging comprising: at least one image sensor (2) whose pixels are based on photodiodes; wherein the image recording device (1) has a single photon-sensitive detector (SPSD) (3) and the image sensor (2) and the SPSD (3) are arranged to detect light from a common object area (4); wherein the image sensor (2) detects the light from the common object area (4) with the photodiodes; wherein the SPSD (3) is a sensor device separate from the image sensor (2), the SPSD (3) is based on different individual detectors than the photodiodes of the image sensor (2), and the SPSD (3) offers higher sensitivity than the image sensor (2) for detecting the light from the common object area (4); wherein the image recording device (1) has at least one optical filter (9) to keep undesired excitation light, used for fluorescence imaging and emanating from the object area (4), away from the SPSD (3). ### Claim 2 (depends on 1) The image recording device (1) according to claim 1, wherein the image sensor (2) and the SPSD (3) are arranged in a fixed spatial relationship to one another, with respect to a beam splitter (6); and wherein the beam splitter (6) passes light from the common object area (4) to the image sensor (2) and the SPSD (3), and wherein the image sensor (2) and the SPSD (3) are arranged on exterior surfaces of the beam splitter (6) and/or correspond to a common intermediate image plane of an optical imaging unit (7). ### Claim 3 (depends on 1) The image recording device (1) according to claim 1, wherein the image recording device (1) has a common optical imaging unit (7) arranged in a distal end area (5) of the image recording device (1), which conducts light from the common object area (4) to the image sensor (2) and to the SPSD (3). ### Claim 4 (depends on 1) The image recording device (1) according to claim 1, wherein the image recording device (1) forms two separate light guide channels, which each conduct light from the common object area (4) to the image sensor (2) or to the SPSD (3). ### Claim 5 (depends on 1) The image recording device (1) according to claim 1, wherein the image sensor (2) outputs a black and white image or a color image via a color filter array (CFA). ### Claim 6 (depends on 1) The image recording device (1) according to claim 1, wherein the image sensor (2) detects wavelengths in the visible range or alternatively wavelengths in adjacent wavelength ranges, in particular in the ultraviolet or infrared wavelength range. ### Claim 7 (depends on 1) The image recording device (1) according to claim 1, wherein the SPSD (3) is formed as a 2D arrangement of individual detectors that are fully-integrated and/or realized on the basis of semiconductor technology and can respectively detect individual photons so that the SPSD (3) provides 2D image data. ### Claim 8 (depends on 1) The image recording device (1) according to claim 1, wherein the SPSD (3) is implemented based on single photon avalanche diodes (SPADs) and/or silicon photomultipliers (SiPMs). ### Claim 9 (depends on 8) The image recording device (1) of claim 8 wherein the SPSD (3) is implemented as a 2D SPAD array or as a 2D SiPM array. ### Claim 10 (depends on 1) The image recording device (1) of claim 1, wherein the image sensor (2) and/or the SPSD (3) are implemented by means of CMOS technology as fully integrated electronic components, in particular with integrated signal processing electronics. ### Claim 11 (depends on 1) The image recording device of claim 1, wherein the at least one optical filter (9) keeps the undesired excitation light away from the image sensor (2). ### Claim 12 (depends on 1) The image recording device of claim 1, wherein the optical filter (9) selects a wavelength range captured by the SPSD (3) and/or the image sensor (2) and is implemented on the beam splitter (6) as an optical thin film (10). ### Claim 13 (depends on 1) The image recording device (1) according to claim 1, wherein the at least one optical filter (9) is actively tunable such that different wavelengths can be detected by the SPSD (3) depending on the tuning of the optical filter (9). ### Claim 14 (depends on 1) The image recording device (1) according to claim 1, wherein the image recording device (1) has at least two separately arranged SPSDs (3) that detect different wavelength ranges via two different optical filters (9). ### Claim 15 (depends on 1) The image recording device (1) according claim 1, wherein the image recording device (1) has at least two image sensors (2) arranged separately from each other that detect different wavelength ranges via different color filters at the pixel level. ### Claim 16 (depends on 1) The image recording device (1) according to claim 1, wherein the image recording device (1) has an image computing unit (27) that is designed to calculate and output a synthetic image from signals of the image sensor (2) and from signals of the SPSD (3). ### Claim 17 (depends on 1) A stereo image recording device (28) for generating 3D image data that includes a a stereo image recording device (1) having two image recording devices (1) according to claim 1; and wherein the two image recording devices (1) are arranged for stereoscopic imaging, such that 3D image data can be obtained from a matching object area. ### Claim 18 (depends on 1) An image recording method using an image recording device (1) of claim 1, wherein recording a first image, preferably a color image, by means of an image sensor (2) of the image recording device (1), whose pixels are based on photodiodes; recording a second image using a single photon sensitive detector (SPSD) of the image recording device (1); and calculating and outputting a synthetic image from the first image and the second image, wherein autofluorescent light is detected with the SPSD (3) and wherein excitation light used for fluorescence imaging is separated and filtered from the fluorescent light by an optical filter (9) before the fluorescent light reaches the SPSD (3). ### Claim 19 (depends on 18) The image recording method according to claim 18, wherein the second image is scaled to an image resolution of the first image prior to synthesis. ### Claim 20 (depends on 18) The image recording method according to claim 18, wherein the synthetic image is obtained by uniting the first image with the second image by means of an alpha-blending method, wherein alpha values are taken into account in addition to color information as a metric for the transparency or opacity of respective pixels. ### Claim 21 (depends on 18) The image-recording method according to claim 18, wherein at least a third image in an additional wavelength range, captured via an additional image sensor (2) or an additional SPSD (3) of the image recording device (1), is taken into account during the synthesis of the synthetic image, and the synthetic image is output as a hyperspectral image. ### Claim 22 (depends on 18) The image-recording method according to claim 18, wherein a decay behavior of a fluorescent light source, is detected and evaluated by means of the SPSD (3). ### Claim 23 (depends on 18) The image-recording method according to claim 18 wherein a time of flight measurement is implemented using the SPSD (3) by means of a time-of-flight (ToF) method by which location-resolved depth information is determined from an object area (4) outside of the image recording device (1). ### Claim 24 (depends on 1) An image recording method using an image recording device (1) of claim 1, comprising: the image sensor (2) whose pixels are based on photodiodes; the SPSD (3) offering higher sensitivity than the image sensor (2); wherein in a first step: a first fluorescence image is captured with the SPSD (3) and superimposed on a white light image recorded with the image sensor (2) to enable the detection of weak fluorescent light sources in deeper tissue layers, and the first fluorescence image, due to its low signal intensity, is not visible in the white light image recorded with the second image; and subsequently in a second step: a spatially high-resolution second fluorescence image is captured with the image sensor (2) to enable examination of a fluorescent tissue, which has been detected in the first step, in higher spatial resolution using the image sensor (2).