# US 2021/0344835 A1 · Method for reconstructing an output image and/or a sequence of output images from raw image data, image recording device, image recording system, and overall system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344835 A1 - Document ID: US20210344835A1 - Status: Published application - Kind: A1 - Application number: 17/243,631 - Filed: 2021-04-29 - Published: 2021-11-04 - Earliest claimed: 2020-04-30 - Assignees: Schölly Fiberoptic GmbH - Applicants: SchöLly Fiberoptic GmbH - Inventors: STEFAN SCHRÖER; MATTHIAS KÜHN; ALEXANDER KÖHLER; DANIEL HÄRTER; JOACHIM JAUSS - Web page: https://patentgrep.com/patent/US20210344835A1 ## CPC - A61B 1/00009: {of image signals during a use of endoscope} - A61B 1/00011: {characterised by signal transmission} - A61B 1/00045: {Display arrangement} - G06T 7/0014: {using an image reference approach} - H04N 5/765: Interface circuits between an apparatus for recording and another apparatus - H04N 5/91: Television signal processing therefor - H04N 23/80: Camera processing pipelines; Components thereof - A61B 1/042: {characterised by a proximal camera, e.g. a CCD camera} - G06T 2207/10068: Endoscopic image - G06T 2207/30168: Image quality inspection - H04N 23/555: {for picking-up images in sites, inaccessible due to their dimensions or hazardous conditions, e.g. endoscopes or borescopes} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The invention relates to a method for reconstructing an output image and/or a sequence of output images from raw image data, wherein, in at least one reconstruction chain between a recording of raw image data by means of an image recording device and a display of the output image and/or the sequence of output images on a display unit, at least one reconstruction step for reconstructing the output image and/or the sequence of output images is carried out, and wherein the result of the at least one reconstruction step is discarded when the result is recognized as invalid with respect to the fulfillment of a verification criterion. ## Claims ### Claim 1 A method for reconstructing an output image and/or a sequence of output images from raw image data comprising: processing raw image data in at least two consecutive and/or parallel reconstruction stages that include at least a first reconstruction stage and a second reconstruction stage; performing a verification step on the first and second reconstruction stages; outputting the second reconstruction stage as an output image when the second reconstruction stage, compared to the first reconstruction stage, is recognized as valid in a computer-assisted manner; and outputting the first reconstruction stage as an output image when the second reconstruction stage, compared to the first reconstruction stage, is recognized as invalid in a computer-assisted manner. ### Claim 2 (depends on 1) A method according to claim 1 wherein the raw image data is recorded with an image recording device, in particular an endoscope, wherein an output image is reconstructed by computing units from the raw image data before the output image is displayed on a display unit; and wherein for the reconstruction of the output image, a first computing unit of a first computing instance and a second computing unit of a second computing instance are used, which define the at least two reconstruction stages whereby independent computing processes are carried out in different computing steps; and wherein an image of the first reconstruction stage is reconstructed by the first computing unit and an image of the second reconstruction stage, building upon the image of the first reconstruction stage, is reconstructed by the second computing unit, and that, in the verification step, the image of the second reconstruction stage is verified with respect to the presence of a verification criterion, wherein, as the output image, the image having the highest reconstruction stage for which the verification criterion is valid is displayed. ### Claim 3 (depends on 1) The method according to claim 1 further comprising: discarding the result of the reconstruction when the result is recognized as invalid. ### Claim 4 (depends on 1) The method according to claim 1 wherein the image from the first reconstruction stage is an output image when at least one specific feature is not contained in an image from the second reconstruction stage, and wherein the image from the second reconstruction stage is the output image and/or as part of the sequence of output images when at least one specific feature is contained in the image from the second reconstruction stage. ### Claim 5 (depends on 1) A method according to claim 1 wherein in the verification step, a time measurement is carried out, wherein the image having the highest reconstruction stage is the output image, said image having been reconstructed before the expiry of a definable or predefinable lag time. ### Claim 6 (depends on 2) A method according to claim 2, wherein more than two computing units are used, which define a corresponding number of different reconstruction stages or at least partially redundant reconstruction; and wherein, in the verification step, a feature comparison is carried out between images of the first and second reconstruction stages. ### Claim 7 (depends on 2) A method according to claim 2 wherein the first computing unit is a local computing unit, which is part of the image recording device, and the second computing unit is an external computing unit, which is connected in particular to the image recording device and/or the image recording system via a network infrastructure. ### Claim 8 (depends on 2) A method according to claim 2 wherein the second computing unit is a server that can be used by multiple image recording devices and/or image recording systems, a cloud-based infrastructure that is accessible via the internet, and/or a grid computing infrastructure. ### Claim 9 (depends on 1) A method according to claim 1 wherein in the different reconstruction stages, different algorithms are processed, such as, algorithms for reconstructing one or two or more additional items of information, preferably at least one additional item of information from super resolution, 3D Matching, superimposition of computer tomography data, superimposition of magnetic resonance tomography data, or image correction. ### Claim 10 (depends on 2) A method according to claim 2, wherein the first computing unit processes a basic algorithm, which is required to reconstruct an output image; and wherein at least one specific feature of the image of the first reconstruction stage is contained in or can be allocated to all images of the second reconstruction stage, or a higher reconstruction stage, provided that a correct reconstruction and/or allocation of the images of the second or higher reconstruction stage to the image of the first reconstruction stage has occurred. ### Claim 11 (depends on 4) A method according to claim 4 wherein the specific feature is at least one of a time stamp, image content, and/or specific identifier. ### Claim 12 (depends on 2) A method according to claim 2, characterized in that an assignment of the computing units forming the computing instances to the reconstruction stages is fixed. ### Claim 13 (depends on 1) An image recording device, in particular an endoscope and/or camera head comprising: at least one local computing unit and a connection point for at least one external computing unit; and wherein that the image recording device is set up in order to carry out the method of claim 1. ### Claim 14 (depends on 13) The image recording device of claim 13 further comprising: an external computing unit and/or a display unit. ### Claim 15 (depends on 14) The image recording device of claim 14 wherein the image recording device and external computing unit and/or display unit are arranged peripherally and/or retrievably on demand and are connected to one another via a network infrastructure.