# US 9,521,385 B2 · Image sensor equipped with additional group of selectively transmissive filters for illuminant estimation, and associated illuminant estimation method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,521,385 B2 - Document ID: US9521385B2 - Status: Granted - Kind: B2 - Application number: 14/226,833 - Filed: 2014-03-27 - Published: 2015-10-01 - Granted: 2016-12-13 - Earliest claimed: 2014-03-27 - Assignees: Himax Imaging Limited - Applicants: Himax Imaging Limited - Inventors: Po-Chang Chen; Yuan-Chih Peng - Web page: https://patentgrep.com/patent/US9521385B2 ## CPC - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - G02B 5/201: {in the form of arrays} - H04N 25/134: {based on three different wavelength filter elements} - H04N 25/135: {based on four or more different wavelength filter elements} - H04N 25/61: the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4" - H04N 25/703: SSIS architectures incorporating pixels for producing signals other than image signals - G01J 2003/2806: {Array and filter array} - H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values} - H04N 2209/047: using multispectral pick-up elements - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N) ## Abstract An image sensor includes a two dimensional array of pixel elements, a color filter array, and a digital circuit. The color filter array is superimposed on and in registration with the two dimensional array of pixel elements. The color filter array includes a first group of selectively transmissive filters and a second group of selectively transmissive filters. The first group of selectively transmissive filters is arranged to selectively transmit spectral energy in M colors of the visible spectrum, wherein M is larger than two. The second group of selectively transmissive filters is arranged to selectively transmit spectral energy in N colors of the visible spectrum, wherein any selectively transmissive filter of the first and the second groups of selectively transmissive filters is arranged to transmit spectral energy in one specific color of the visible spectrum, and N is larger than M. An associated illuminant estimation method is also provided. ## Claims ### Claim 1 An image sensor, comprising: a two dimensional array of pixel elements, arranged to receive light energy of incident light and to convert the received light energy into electrical signals; a color filter array, superimposed on and in registration with the two dimensional array of pixel elements, the color filter array comprising: a first group of selectively transmissive filters, arranged to selectively transmit spectral energy in M colors of the visible spectrum, wherein M is a positive integer that is larger than two; and a second group of selectively transmissive filters, arranged to selectively transmit spectral energy in N colors of the visible spectrum, wherein any selectively transmissive filter of the first and the second groups of selectively transmissive filters is arranged to transmit spectral energy in one specific color of the visible spectrum, and N is a positive integer that is larger than M; and a digital circuit, coupled to the two dimensional array of pixel elements, configured to generate a plurality of output signals for pixel elements within the two dimensional array of pixel elements, wherein at least one portion of the output signals carries pixel data representing an image of a scene; wherein the image of the scene corresponds to raw image data of pixel elements corresponding to the first group of selectively transmissive filters, and correction of the image is performed according to information derived from pixel elements corresponding to the second group of selectively transmissive filters; and the digital circuit operates according to an illuminant estimation method for the image sensor, the color filter array comprises a multi-band color filter array (CFA), and the illuminant estimation method comprises the steps of: determining a plurality of spectral ratio vectors associated with a plurality of pre-defined light sources, respectively; receiving a series of multi-band CFA pixel data from the image sensor; computing a mean value for each spectral channel of a plurality of spectral channels and converting mean values respectively corresponding to the plurality of spectral channels into vector elements of a current spectral ratio vector; comparing the current spectral ratio vector with the spectral ratio vectors of the pre-defined light sources and computing a norm of the current spectral ratio vector and each of the spectral ratio vectors of the pre-defined light sources, to determine a specific pre-defined light source whose spectral ratio vector is closest to the current spectral ratio vector within the plurality of pre-defined light sources; and outputting a detection result indicating the specific pre-defined light source whose spectral ratio vector is closest to the current spectral ratio vector within the plurality of pre-defined light sources. ### Claim 2 (depends on 1) The image sensor of claim 1, wherein the second group of selectively transmissive filters is disposed and arranged at one side of the first group of selectively transmissive filters. ### Claim 3 (depends on 2) The image sensor of claim 2, wherein the one side of the first group of selectively transmissive filters is a top side of the first group of selectively transmissive filters. ### Claim 4 (depends on 3) The image sensor of claim 3, wherein for the image of the scene, at least one output signal of at least one pixel element corresponding to the top side of the first group of selectively transmissive filters is output before at least one output signal of at least one pixel element corresponding to a bottom side of the first group of selectively transmissive filters is output, wherein the bottom side is opposite to the top side. ### Claim 5 (depends on 2) The image sensor of claim 2, wherein the digital circuit performs illuminant estimation according to pixel data generated for pixel elements corresponding to the second group of selectively transmissive filters, to generate at least one illuminant estimation result. ### Claim 6 (depends on 5) The image sensor of claim 5, further comprising a third group of non-transmissive filters, wherein before performing the illuminant estimation, the digital circuit performs black level estimation to obtain a black level according to pixel data generated for pixel elements corresponding to the third group, and performs black level correction of both the pixel data generated for the pixel elements corresponding to the second group of selectively transmissive filters and the pixel data generated for the pixel elements corresponding to the first group of selectively transmissive filters according to the black level. ### Claim 7 (depends on 1) The image sensor of claim 1, wherein the digital circuit performs color correction or lens shading correction on pixel data generated for pixel elements corresponding to the first group of selectively transmissive filters, to generate the pixel data representing the image of the scene. ### Claim 8 (depends on 1) The image sensor of claim 1, wherein at least one portion of the second group of selectively transmissive filters is disposed and arranged in at least one area enclosed by the first group of selectively transmissive filters. ### Claim 9 (depends on 1) The image sensor of claim 1, wherein for the image of the scene, at least one output signal of at least one pixel element corresponding to the first group of selectively transmissive filters is output after at least one output signal of at least one pixel element corresponding to the second group of selectively transmissive filters is output. ### Claim 10 An illuminant estimation method for an image sensor, the image sensor comprising a multi-band color filter array (CFA), the illuminant estimation method comprising the steps of: determining a plurality of spectral ratio vectors associated with a plurality of pre-defined light sources, respectively; receiving a series of multi-band CFA pixel data from the image sensor; computing a mean value for each spectral channel of a plurality of spectral channels and converting mean values respectively corresponding to the plurality of spectral channels into vector elements of a current spectral ratio vector; comparing the current spectral ratio vector with the spectral ratio vectors of the pre-defined light sources and computing a norm of the current spectral ratio vector and each of the spectral ratio vectors of the pre-defined light sources, to determine a specific pre-defined light source whose spectral ratio vector is closest to the current spectral ratio vector within the plurality of pre-defined light sources; and outputting a detection result indicating the specific pre-defined light source whose spectral ratio vector is closest to the current spectral ratio vector within the plurality of pre-defined light sources. ### Claim 11 (depends on 10) The illuminant estimation method of claim 10, wherein right after computing the mean for each spectral channel of the plurality of spectral channels and converting the spectral channel mean values into the vector elements of the current spectral ratio vector, the current spectral ratio vector is multiplied by a spectral conversion vector so that one or more spectral bands of interest are further weighted. ### Claim 12 (depends on 10) An apparatus that operates according to the method of claim 10, wherein the apparatus comprises the image sensor. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the image sensor comprises: a two dimensional array of pixel elements, arranged to receive light energy of incident light and to convert the received light energy into electrical signals; a color filter array, superimposed on and in registration with the two dimensional array of pixel elements, the color filter array comprising: a first group of selectively transmissive filters, arranged to selectively transmit spectral energy in M colors of the visible spectrum, wherein M is a positive integer that is larger than two; and a second group of selectively transmissive filters, arranged to selectively transmit spectral energy in N colors of the visible spectrum, wherein any selectively transmissive filter of the first and the second groups of selectively transmissive filters is arranged to transmit spectral energy in one specific color of the visible spectrum, and N is a positive integer that is larger than M; and a digital circuit, coupled to the two dimensional array of pixel elements, configured to generate a plurality of output signals for pixel elements within the two dimensional array of pixel elements, wherein at least one portion of the output signals carries pixel data representing an image of a scene; wherein the color filter array comprises the multi-band color filter array (CFA).