# US 12,120,472 B2 · High sensitivity color image generation using hybrid sensor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,120,472 B2 - Document ID: US12120472B2 - Status: Granted - Kind: B2 - Application number: 17/691,043 - Filed: 2022-03-09 - Published: 2022-09-22 - Granted: 2024-10-15 - Earliest claimed: 2021-03-17 - Applicants: MediaTek Inc. - Inventors: Yu-Ju Lin; Ying-Jui Chen; Keh-Tsong Li; Pin-Chung Lin; Hung-Chih Ko; Chi-Cheng Ju - Web page: https://patentgrep.com/patent/US12120472B2 ## CPC - H04N 9/646: {for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters} - H04N 25/133: including elements passing panchromatic light, e.g. filters passing white light - H04N 5/30: Transforming light or analogous information into electric information (scanning details H04N3/00; cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00; circuitry of solid-state image sensors [SSIS] or control thereof H04N25/00) - H04N 23/11: for generating image signals from visible and infrared light wavelengths - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Various schemes pertaining to generating a full-frame color image using a hybrid sensor are described. An apparatus receives sensor data from the hybrid sensor, wherein the sensor data includes partial-frame chromatic data of a plurality of chromatic channels and partial-frame color-insensitive data. The apparatus subsequently generates full-frame color-insensitive data based on the partial-frame color-insensitive data. The apparatus subsequently generates the full-frame color image based on the full-frame color-insensitive data and the partial-frame chromatic data. The apparatus provides benefits of enhancing image quality of the full-frame color image especially under low light conditions. ## Claims ### Claim 1 A method of generating a full-frame color image, comprising: receiving sensor data from a hybrid sensor, the sensor data comprising partial-frame chromatic data of a plurality of chromatic channels and partial-frame color-insensitive data; generating full-frame color-insensitive data based on the partial-frame color-insensitive data; and generating the full-frame color image based on the full-frame color-insensitive data and the partial-frame chromatic data, wherein the full-frame color image comprises a plurality of full-frame chromatic images each corresponding to a respective one of the plurality of chromatic channels, and wherein the generating of the full-frame color image comprises generating each of the plurality of full-frame chromatic images using a colorization process based on the full-frame color-insensitive data and the partial-frame chromatic data of the respective chromatic channel. ### Claim 2 (depends on 1) The method of claim 1, wherein: the hybrid sensor comprises a sensor array having a plurality of chromatic sensing elements of a first quantity and a plurality of non-chromatic sensing elements of a second quantity greater than the first quantity, the plurality of chromatic sensing elements configured to generate the partial-frame chromatic data, the plurality of non-chromatic sensing elements configured to generate the partial-frame color-insensitive data, and the plurality of chromatic sensing elements are sparsely distributed among the plurality of non-chromatic sensing elements across a light-receiving surface of the sensor array. ### Claim 3 (depends on 1) The method of claim 1, wherein the generating of the full-frame color-insensitive data comprises interpolating the partial-frame color-insensitive data. ### Claim 4 (depends on 1) The method of claim 1, wherein the generating of the full-frame color-insensitive data is further based on the partial-frame chromatic data. ### Claim 5 (depends on 1) The method of claim 1, wherein the colorization process comprises applying the partial-frame chromatic data of the respective chromatic channel to the full-frame color-insensitive data through a substantially linear function. ### Claim 6 (depends on 5) The method of claim 5, wherein the colorization process further comprises determining one or more coefficients of the substantially linear function using a guided filter solver or a bilateral solver. ### Claim 7 (depends on 1) The method of claim 1, wherein the plurality of chromatic channels comprises a red channel, a green channel and a blue channel. ### Claim 8 (depends on 1) The method of claim 1, wherein the plurality of chromatic channels comprises a cyan channel, a magenta channel and a yellow channel. ### Claim 9 (depends on 1) The method of claim 1, wherein the partial-frame color-insensitive data comprises white light data, infrared data, or a combination of both. ### Claim 10 An apparatus, comprising: an input module configured to receive sensor data from a hybrid sensor, the sensor data comprising partial-frame chromatic data of a plurality of chromatic channels and partial-frame color-insensitive data; a spatial filtering module configured to generate full-frame color-insensitive data based on the partial-frame color-insensitive data; and a colorization module configured to generate a plurality of full-frame chromatic images based on the full-frame color-insensitive data and the partial-frame chromatic data, each of the plurality of full-frame chromatic images corresponding to a respective one of the plurality of chromatic channels, wherein the colorization module is, in generating the plurality of full-frame chromatic images, further configured to apply the partial-frame chromatic data of the each of the plurality of chromatic channels to the full-frame color-insensitive data through a respective linear function. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein: the input module is further configured to receive the sensor data from a hybrid sensor, the hybrid sensor comprises a sensor array having a plurality of chromatic sensing elements of a first quantity and a plurality of non-chromatic sensing elements of a second quantity greater than the first quantity, the plurality of chromatic sensing elements configured to generate the partial-frame chromatic data, the plurality of non-chromatic sensing elements configured to generate the partial-frame color-insensitive data, and the plurality of chromatic sensing elements are sparsely distributed among the plurality of non-chromatic sensing elements across a light-receiving surface of the sensor array. ### Claim 12 (depends on 10) The apparatus of claim 10, wherein the spatial filtering module is configured to generate the full-frame color-insensitive data by performing interpolation on the partial-frame color-insensitive data. ### Claim 13 (depends on 10) The apparatus of claim 10, wherein the spatial filtering module is configured to generate the full-frame color-insensitive data further based on the partial-frame chromatic data. ### Claim 14 (depends on 10) The apparatus of claim 10, further comprising: an output module configured to transmit the plurality of full-frame chromatic images as a full-frame color image. ### Claim 15 (depends on 10) The apparatus of claim 10, wherein the colorization module is further configured to determine one or more coefficients of the linear function using a guided filter solver or a bilateral solver. ### Claim 16 (depends on 10) The apparatus of claim 10, wherein the plurality of chromatic channels comprises a red channel, a green channel and a blue channel. ### Claim 17 (depends on 10) The apparatus of claim 10, wherein the plurality of chromatic channels comprises a cyan channel, a magenta channel and a yellow channel. ### Claim 18 (depends on 10) The apparatus of claim 10, wherein the partial-frame color-insensitive data comprises white light data, infrared data, or a combination of both.