# US 11,902,676 B2 · Image signal processor, operation method of image signal processor, and image sensor device including image signal processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,902,676 B2 - Document ID: US11902676B2 - Status: Granted - Kind: B2 - Application number: 17/859,778 - Filed: 2022-07-07 - Published: 2023-01-12 - Granted: 2024-02-13 - Earliest claimed: 2021-07-07 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Seungwon Choi; Ildo Kim; Jongseong Choi; Ji Hwan Park; Ser Hoon Lee; Sukhwan Lim; Kyuyul Choi - Web page: https://patentgrep.com/patent/US11902676B2 ## CPC - H04N 23/45: for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images - H04N 25/46: by combining or binning pixels - G06T 5/50: using two or more images, e.g. averaging or subtraction - H04N 23/665: {involving internal camera communication with the image sensor, e.g. synchronising or multiplexing SSIS control signals} - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 25/75: Circuitry for providing, modifying or processing image signals from the pixel array - G06T 2207/10024: Color image - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Provided are image signal processing apparatuses and methods for operating the same. In an embodiment, the image signal processing apparatus is configured to receive, from an image sensor device, an input image. The apparatus is further configured to perform a binning and crop operation on the input image to generate a first image. The apparatus is further configured to perform a Bayer domain processing on the first image to generate a second image. The apparatus is further configured to perform RGB domain processing on the second image to generate a third image. The apparatus is further configured to perform YUV domain processing on the third image to generate an output image. The YUV domain processing comprises at least one of a spatial de-noising operation, a temporal de-noising operation, a motion compensation operation, a tone mapping operation, a detail enhance operation, and a sharpening operation. ## Claims ### Claim 1 An image signal apparatus, comprising: a memory storage; and an image signal processor communicatively coupled to the memory storage, wherein the image signal processor is configured to cause the image signal apparatus to: receive, from a first image sensor device, a first input image; perform a binning and crop operation on the first input image to generate a first image; generate first frame information about the first input image based on the first image, wherein the first frame information comprises image pyramid information about at least a portion of a first frame of the first image; perform a Bayer domain processing on the first image to generate a second image; perform RGB domain processing on the second image to generate a third image; and perform YUV domain processing on the third image to generate an output image, wherein the YUV domain processing comprises at least one of a spatial de-noising operation, a temporal de-noising operation, a motion compensation operation, a tone mapping operation, a detail enhance operation, and a sharpening operation. ### Claim 2 (depends on 1) The image signal apparatus of claim 1, wherein, a color pattern of the first input image matches a color pattern of a color filter array included in the first image sensor device, the first image has a Bayer color pattern, the second image has an RGB color pattern, and the third image has a YUV color pattern. ### Claim 3 (depends on 1) The image signal apparatus of claim 1, wherein a resolution of the second image matches a resolution of the third image. ### Claim 4 (depends on 1) The image signal apparatus of claim 1, wherein a resolution of the second image matches a resolution of the output image. ### Claim 5 (depends on 1) The image signal apparatus of claim 1, wherein the image signal processor is further configured to: receive, from a binning and crop module, the first image. ### Claim 6 (depends on 5) The image signal apparatus of claim 5, wherein the first frame information comprises at least one of image pyramid information about one or more areas of the first image, average information for each area of the one or more areas, distribution information for each area of the one or more areas, histogram information for each area of the one or more areas, and motion information. ### Claim 7 (depends on 6) The image signal apparatus of claim 6, wherein, the first frame information is stored in an external memory device, and wherein the image signal processor is further configured to: perform the Bayer domain processing on the first image, using the first frame information stored in the external memory device, to generate the second image; perform the RGB domain processing on the second image, using the first frame information stored in the external memory device, to generate the third image; and perform the YUV domain processing on the third image, using the first frame information stored in the external memory device, to generate the output image. ### Claim 8 (depends on 5) The image signal apparatus of claim 5, wherein the image signal processor is further configured to: receive, from a second image sensor device, a second input image; and generate second frame information about the second input image. ### Claim 9 (depends on 1) The image signal apparatus of claim 1, wherein to perform the temporal de-noising operation on the third image, the image signal processor is configured to store the third image in an external memory device and to access the third image stored in the external memory device. ### Claim 10 (depends on 1) The image signal apparatus of claim 1, wherein the first image is stored in an external memory device, and wherein the image signal processor accesses the external memory device to receive the first image. ### Claim 11 An operation method of an image signal processor, the operation method comprising: receiving, from an image sensor device, an input image; performing a binning and crop operation on the input image to generate a first image; generating frame information about the first input image based on the first image, wherein the frame information comprises image pyramid information about at least a portion of a first frame of the first image; performing Bayer domain processing on the first image to generate a second image; performing RGB domain processing on the second image to generate a third image; and performing YUV domain processing on the third image to generate an output image, wherein the performing of the YUV domain processing comprises outputting the output image, wherein a resolution of the second image matches a resolution of the third image, and wherein the YUV domain processing comprises at least one of a spatial de-noising operation, a temporal de-noising operation, a motion compensation operation, a tone mapping operation, a detail enhance operation, and a sharpening operation. ### Claim 12 (depends on 11) The operation method of claim 11, wherein, a color pattern of the input image matches a color pattern of a color filter array included in the image sensor device, the first image has a Bayer color pattern, the second image has an RGB color pattern, and the third image has a YUV color pattern. ### Claim 13 (depends on 11) The operation method of claim 11, wherein the resolution of the second image matches the resolution of the third image. ### Claim 14 (depends on 11) The operation method of claim 11, wherein the frame information further comprises at least one of image pyramid information about one or more areas of the first image, average information for each area of the one or more areas, distribution information for each area of the one or more areas, histogram information for each area of the one or more areas, and motion information. ### Claim 15 (depends on 14) The operation method of claim 14, wherein, the frame information is stored in an external memory device, the performing of the Bayer domain processing on the first image comprises performing the Bayer domain processing on the first image, using the frame information stored in the external memory device, to generate the second image, the performing of the RGB domain processing on the second image comprises performing the RGB domain processing on the second image, using the frame information stored in the external memory device, to generate the third image, and the performing of the YUV domain processing on the third image comprises performing the YUV domain processing on the third image, using the frame information stored in the external memory device, to generate the output image is performed by using the frame information stored in the external memory device. ### Claim 16 (depends on 14) The operation method of claim 14, wherein, the first image is stored in an external memory device, and the performing of the Bayer domain processing on the first image comprises accessing the external memory device to obtain the first image. ### Claim 17 An image sensor device, comprising: a pixel array comprising a plurality of pixels; a row driver configured to control the plurality of pixels; an analog-to-digital converter configured to convert pixel signals into digital signals; an output buffer configured to output an input image based on the digital signals; and an image signal processor configured to generate an output image based on the input image, wherein the image signal processor is further configured to: generate first frame information based on the input image; perform a binning and crop operation on the input image to generate a first image; generate second frame information about the first input image based on the first image, wherein the second frame information comprises image pyramid information about at least a portion of a first frame of the first image; perform Bayer domain processing on the first image based on the first frame information to generate a second image; perform RGB domain processing on the second image based on the first frame information to generate a third image; and perform YUV domain processing on the third image based on the first frame information to generate the output image, wherein the YUV domain processing comprises at least one of a spatial de-noising operation, a temporal de-noising operation, a motion compensation operation, a tone mapping operation, a detail enhance operation, and a sharpening operation. ### Claim 18 (depends on 17) The image sensor device of claim 17, wherein the image signal processor is further configured to: receive, from a binning and crop module, the first image. ### Claim 19 (depends on 18) The image sensor device of claim 18, wherein the second frame information further comprises at least one of image pyramid information about one or more areas of the first image, average information for each area of the one or more areas, distribution information for each area of the one or more areas, histogram information for each area of the one or more areas, and motion information. ### Claim 20 (depends on 18) The image sensor device of claim 18, wherein the image signal processor is further configured to prevent, when the image sensor device is a slave camera, performance of the binning and crop operation, the Bayer domain processing, the RGB domain processing, and the YUV domain processing.