# US 8,542,299 B2 · Image processing device, imaging device, image processing method, imaging method, and image processing program for pseudo-color suppression in an image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 8,542,299 B2 - Document ID: US8542299B2 - Status: Granted - Kind: B2 - Application number: 13/357,284 - Filed: 2012-01-24 - Published: 2012-07-12 - Granted: 2013-09-24 - Earliest claimed: 2007-07-13 - Assignees: Intel Benelux B.V. - Applicants: Yasuhiro Sawada - Inventors: Yasuhiro Sawada - Web page: https://patentgrep.com/patent/US8542299B2 ## CPC - H04N 9/646: {for image enhancement, e.g. vertical detail restoration, cross-colour elimination, contour correction, chrominance trapping filters} - G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns - H04N 23/12: with one sensor only - H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values} - H04N 25/134: {based on three different wavelength filter elements} - H04N 25/704: Pixels specially adapted for focusing, e.g. phase difference pixel sets - H04N 2209/046: Colour interpolation to calculate the missing colour values - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image processing device comprising a synchronization unit (25) for generating a luminance (Y′) from the sum of pixel signals R, Gr, Gb, B, for subtracting the R pixel signal and the B pixel signal from the sum of the Gr pixel signal and Gb pixel signal so as to generate a first color difference (C1), and for calculating a difference between the R pixel signal and the B pixel signal to generate a second color difference (C2), a pseudo-color suppression unit (31) for performing pseudo-color suppression of the first color difference (C1) and/or the second color difference (C2), a color space conversion unit (37) for converting the luminance Y′, the first color difference (C1), the second color difference (C2), into a predetermined color space to generate YUV color information. ## Claims ### Claim 1 An image processing device for generating a color image, having a plurality of color information for each pixel, from a color mosaic image obtained from an image sensor with a matrix of units, a unit comprising one or more R (red), one or more G (green) and one or more B (blue) pixels, and wherein, at the pixel position in the above mentioned-color image corresponding to an above mentioned unit, a 1st color difference is generated by subtracting the sum of respective-R pixel signals for said R pixels in said unit and of respective B pixel signals for said one or more B pixels in said unit from the sum of respective G pixel signals for said one or more G pixels in said unit, the image processing device comprising a synchronization facility for generating a 2nd color difference by calculating the difference between the sum of the above mentioned respective R pixel signals and the sum of the above mentioned respective B pixel signals, and the image processing device comprising a pseudo-color suppression facility for suppressing pseudo-color in at least one of the above mentioned 1st color difference and 2nd color difference, a color space conversion facility for converting the pseudo-color suppressed 1st color difference and 2nd color difference obtained from the pseudo-color suppression facility using said luminance into a determined color space and for generating the above mentioned color information. ### Claim 2 (depends on 1) The image processing device of claim 1, provided with a pseudo-color information generation facility, that generates at least one of a 1st parameter or 2nd parameter expressing the high frequency component amount of a luminance in mutually differing directions. ### Claim 3 (depends on 2) The image processing device of claim 2, wherein the unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), and wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal, wherein the 2nd color difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal, and wherein the above mentioned 2nd parameter is a parameter that expresses a difference between the above mentioned Gr pixel signal and the Gb pixel signal. ### Claim 4 (depends on 2) The image processing device of claim 2 wherein the above mentioned pseudo-color suppression facility is arranged to suppress the above mentioned 1st color difference in accordance with the above mentioned 1st parameter and is arranged to suppress the above mentioned 2nd color difference in accordance with the above mentioned 2nd parameter. ### Claim 5 (depends on 1) The image processing device of claim 1 having a low frequency pass filter for removal of the high frequency component in the 1st color difference and 2nd color difference obtained from the above mentioned pseudo-color suppression facility. ### Claim 6 (depends on 1) The image processing device of claim 1 wherein the above mentioned pseudo-color suppression facility is a low frequency pass filter for removal of the high frequency component of the 1st color difference and 2nd color difference. ### Claim 7 (depends on 1) An imaging device comprising an image processing device according to claim 1 further comprising an imaging optics system to image an object image to the image sensor and wherein the above mentioned image processing device generates a color image having the multiple color information of each pixel in accordance with the mosaic image signal output from the above mentioned image sensor. ### Claim 8 (depends on 7) The imaging device of claim 7 being equipped with an optical low-pass filter for suppression of luminance high frequency of an oblique direction on the above mentioned color image pixel array, the optical low-pass filter being arranged in the incident light path forming the object image on the above mentioned image sensor, wherein a unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), and wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal wherein the 2nd difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal, and wherein the cutoff of the aforementioned optical low pass filter being associated with the above mentioned color mosaic image pixel array direction, forming a quadrangle having a 45° angle in the above mentioned color mosaic image frequency space and with the origin point of said space as its center, with the inside of this quadrangle being the transmission band. ### Claim 9 (depends on 1) The image processing device according to claim 1, wherein a unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), and wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal and wherein the 2nd color difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal. ### Claim 10 (depends on 9) The image processing device according to claim 9, wherein the image sensor is equipped with a Bayer filter. ### Claim 11 (depends on 10) The image processing device according to claim 10, wherein a luminance (Y) is generated from the sum of the above mentioned R, Gr, Gb and B pixel signals. ### Claim 12 An image processing method for generating a color image having a plurality of color information for each pixel from a color mosaic image from an image sensor with a matrix of units, a unit comprising one or more R (red), one or more G (green) and one or more B (blue) pixels, said method comprising, at the pixel position in the above mentioned color image corresponding to an above mentioned unit, a 1st color difference generation step for generating a first color difference wherein a sum of respective R pixel signals for said one or more R pixels in said unit and of respective B pixel signals for said one or more B pixels in said unit is subtracted from the sum of respective G pixel signals for said one or more G pixels in said unit, a 2nd color difference generation step for generating a second color difference wherein the difference between the sum of the one or more R pixel signals and the sum of the one or more B pixel signals is calculated, a pseudo-color suppression step for suppressing pseudo-color in at least one of the above mentioned 1st color difference and 2nd color difference, and a color space conversion step for converting said 1st color difference and 2nd color difference for which said pseudo-color suppression step using a luminance was applied into a predetermined color space and for generating the above mentioned color information. ### Claim 13 (depends on 12) The image processing method according to claim 12 comprising a pseudo-color information generation step involving generation of at least one parameter of a 1st or a 2nd parameter expressing the high frequency component amount of the luminance in mutually differing directions. ### Claim 14 (depends on 13) The image processing method according to claim 13, wherein a unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), and wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal wherein the 2nd color difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal, and wherein said 2nd parameter is a parameter expressing a difference between the above mentioned Gr pixel signal and Gb pixel signal. ### Claim 15 (depends on 13) The image processing method according to claim 13 wherein in said pseudo-color suppression step, the above mentioned 1st color difference pseudo-color is suppressed in accordance with the above mentioned 1st parameter and the above mentioned 2nd color difference pseudo-color is suppressed in accordance with the above mentioned 2nd parameter. ### Claim 16 (depends on 12) The image processing method according claim 12 wherein a low pass filter is used to remove a high frequency component from the pseudo color suppressed 1st color difference and 2nd color difference obtained from the above mentioned pseudo-color suppression step. ### Claim 17 (depends on 12) The image processing method according to claim 12 wherein in said pseudo-color suppression step the 1st color difference and 2nd color difference high frequency component is removed with the use of a low frequency pass filter. ### Claim 18 (depends on 12) An imaging method comprising the said image processing method according to claim 12 wherein a color image having a plurality of color information for each pixel is generated in accordance with the image signal of the color mosaic image output, which uses a photo-optical system to introduce the image, from the above mentioned image sensor. ### Claim 19 (depends on 18) The imaging method according to claim 18 wherein a unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal, and wherein the 2nd color difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal, and wherein in the incident light path forming the object image on the above mentioned image sensor, the above mentioned color image pixel array is equipped with an optical low pass filter for suppression of high frequency luminance in an oblique direction with the cutoff of the aforementioned optical low pass filter being associated with the above mentioned color mosaic image pixel array direction, forming a quadrangle having a 45° angle in said color mosaic image frequency space with the origin point of said space as its center on, with the inside of this quadrangle being the transmission band. ### Claim 20 A non-transitory computer-readable medium containing an image processing program for computer execution wherein: a color image having a plurality of color information for each pixel from a color mosaic image obtained from an image sensor is generated, the image sensor comprising a matrix of units, a unit comprising one or more R (red), one or more G (green) and one or more B (blue) pixels, and at the pixel position in the above mentioned color image corresponding to a unit, the program comprising a 1st color difference generation step for generating a first color difference, by subtracting a sum of respective R pixel signals for said one or more R pixels in said unit and of respective B pixel signals for said one or more B pixels in said unit from the sum of respective B pixel signals for said one or more B pixels in said unit and, a 2nd color difference generation step for generating a 2nd color difference, by calculation of the difference between the sum of the above mentioned respective R pixel signals and the sum of the above-mentioned B pixel signals and, a pseudo-color suppression step for suppressing pseudo-color in at least one of said 1st color difference and said 2nd color difference, a color space conversion step for converting the pseudo-color suppressed 1st color difference and 2nd color difference, having gone through the above-mentioned pseudo-color suppression step using a luminance, into a determined color space and for generating the above mentioned color information. ### Claim 21 (depends on 20) The non-transitory computer-readable medium containing the image processing program according to claim 20 that causes a computer to perform a pseudo-color information generation step that generates as the said pseudo-color suppression information at least one parameter of a 1st or 2nd parameter expressing the high frequency component amount of the luminance in mutually different directions. ### Claim 22 (depends on 21) The non-transitory computer-readable medium containing the image processing program according to claim 21, wherein a unit comprises a R (red) pixel, a B (blue) pixel, a Gr pixel (a green pixel next to the above mentioned R pixel in one direction of the pixel array), and a Gb pixel (a green pixel next to the B pixel in the above mentioned one direction), and wherein the 1st color difference is generated by subtracting the above mentioned R pixel signal and B pixel signal from the sum of the above mentioned Gr pixel signal and Gb pixel signal, wherein the 2nd color difference is generated by calculating the difference between the above mentioned R pixel signal and B pixel signal, and wherein the above mentioned 2nd parameter is the parameter expressing the difference between the above mentioned Gr pixel signal and the above mentioned Gb pixel signal. ### Claim 23 (depends on 21) The non-transitory computer-readable medium containing the image processing program according to claim 21, wherein, in the above mentioned pseudo-color suppression, the pseudo-color of the above mentioned 1st color difference is suppressed in accordance with the above mentioned 1st parameter and the pseudo-color of the 2nd color difference is suppressed in accordance with the above mentioned 2nd parameter by execution of the computer program.