# US 12,217,433 B2 · Adaptive auto white balancing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,433 B2 - Document ID: US12217433B2 - Status: Granted - Kind: B2 - Application number: 17/940,891 - Filed: 2022-09-08 - Published: 2024-03-14 - Granted: 2025-02-04 - Earliest claimed: 2022-09-08 - Assignees: Cista System Corp. - Applicants: Cista System Corp. - Inventors: Yang-Ting Chou; Zhaojian Li - Web page: https://patentgrep.com/patent/US12217433B2 ## CPC - G06T 7/11: Region-based segmentation - G06T 3/4015: Image demosaicing, e.g. colour filter arrays [CFA] or Bayer patterns - G06T 5/20: using local operators - G06T 7/90: Determination of colour characteristics - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - G06T 2207/20081: Training; Learning - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract This application describes systems and methods for detecting depth in deep trench isolation with semiconductor devices using test key transistors. An method comprises: capturing, by an image sensor, an image; generating a plurality of chrominance channels by converting the image into luminance-chrominance space; performing homogeneous region segmentation on the plurality of chrominance channels to generate one or more regions of interest in the plurality of chrominance channels; and projecting the regions of interest onto eigen-illuminant images to determine gray color pixels on the image, wherein the eigen-illuminant images are generated via performing a machine learning algorithm on a training set of images captured by the image sensor. ## Claims ### Claim 1 An image sensor apparatus, comprising: a memory configured to store eigen-illuminant images; an image sensing module configured to capture an image; and an image signal processor configured to: generate a plurality of chrominance channels by converting the image into luminance-chrominance space; perform homogeneous region segmentation on the plurality of chrominance channels to generate one or more regions of interest in the plurality of chrominance channels; receive the eigen-illuminant images from the memory, and project the regions of interest onto eigen-illuminant images to determine gray color pixels on the image, wherein the eigen-illuminant images are generated via performing a machine learning algorithm on a training set of images captured by the image sensor. ### Claim 2 (depends on 1) The image sensor apparatus of claim 1, wherein: the image sensing module is further configured to capture the image through a color filter array, and the image signal processor is further configured to: demosaic the captured image to generate a full-color image corresponding to the captured image, and perform low-pass filtering on the plurality of chrominance channels. ### Claim 3 (depends on 1) The image sensor apparatus of claim 1, wherein the eigen-illuminant images are generated by: capturing the training set of images, wherein each of the training set of images include one or more gray cards exposed to one or more environment illuminants; extracting illuminant features in the training set of images using the machine learning algorithm; and generating the eigen-illuminant images according to the extracted illuminant features. ### Claim 4 (depends on 1) The image sensor apparatus of claim 1, wherein the machine learning algorithm includes principal component analysis. ### Claim 5 (depends on 4) The image sensor apparatus of claim 4, wherein: the captured training set of images are demosaiced to generate a plurality of full-color images corresponding to the training set of images, the plurality of full-color images are converted into luminance-chrominance space, and a low-pass filtering is performed on the full-color images in the luminance-chrominance space. ### Claim 6 (depends on 1) The image sensor apparatus of claim 1, wherein the image signal processor is further configured to: determine whether a pixel of the captured image belongs to one of the one or more regions of interest based on local standard deviation of the pixel, and in response to determining that the pixel of the captured image does not belong to any of the one or more regions of interest, set the pixel to null in the plurality of chrominance channels. ### Claim 7 (depends on 1) The image sensor apparatus of claim 1, wherein the image signal processor is further configured to: determine a set of weight corresponding to the eigen-illuminant images, reconstruct the plurality of chrominance channels according to the set of weight, and determine the gray color pixels according to the reconstructed plurality of chrominance channels. ### Claim 8 (depends on 7) The image sensor apparatus of claim 7, wherein the image signal processor is further configured to: determine a white balance gain of the captured image according to the determined gray color pixels, and generate a white balanced image corresponding to the captured image according to the white balance gain. ### Claim 9 (depends on 1) The image sensor apparatus of claim 1, wherein the image signal processor is implemented on an application-specific integrated circuit (ASIC) as a part of the image sensor. ### Claim 10 (depends on 1) The image sensor apparatus of claim 1, wherein the memory is a flash memory. ### Claim 11 A method of auto white balancing, comprising: capturing, by an image sensor, an image; generating a plurality of chrominance channels by converting the image into luminance-chrominance space; performing homogeneous region segmentation on the plurality of chrominance channels to generate one or more regions of interest in the plurality of chrominance channels; and projecting the regions of interest onto eigen-illuminant images to determine gray color pixels on the image, wherein the eigen-illuminant images are generated via performing a machine learning algorithm on a training set of images captured by the image sensor. ### Claim 12 (depends on 11) The method of claim 11, wherein: the image is captured by the image sensor through a color filter array, and the method further comprises: demosaicing the captured image to generate a full-color image corresponding to the captured image. ### Claim 13 (depends on 11) The method of claim 11, further comprising: performing low-pass filtering on the plurality of chrominance channels. ### Claim 14 (depends on 11) The method of claim 11, further comprising: capturing the training set of images, wherein each of the training set of images include one or more gray cards exposed to one or more environment illuminants; extracting illuminant features in the training set of images using the machine learning algorithm; and generating the eigen-illuminant images according to the extracted illuminant features. ### Claim 15 (depends on 14) The method of claim 14, wherein the machine learning algorithm includes principal component analysis. ### Claim 16 (depends on 14) The method of claim 14, further comprising: demosaicing the captured training set of images to generate a plurality of full-color images corresponding to the training set of images; converting the plurality of full-color images into luminance-chrominance space; and performing low-pass filtering on the full-color images in the luminance-chrominance space. ### Claim 17 (depends on 11) The method of claim 11, wherein: performing homogeneous region segmentation on the plurality of chrominance channels to generate the one or more regions of interest in the plurality of chrominance channels further comprises: determining whether a pixel of the captured image belongs to one of the one or more regions of interest based on local standard deviation of the pixel, and the method further comprises: in response to determining that the pixel of the captured image does not belong to any of the one or more regions of interest, setting the pixel to null in the plurality of chrominance channels. ### Claim 18 (depends on 11) The method of claim 11, wherein projecting the regions of interest onto the eigen-illuminant images to determine the gray color pixels on the image further comprises: determining a set of weight corresponding to the eigen-illuminant images, reconstructing the plurality of chrominance channels according to the set of weight, and determining the gray color pixels according to the reconstructed plurality of chrominance channels. ### Claim 19 (depends on 18) The method of claim 18, further comprising: determining a white balance gain of the captured image according to the determined gray color pixels; and generating a white balanced image corresponding to the captured image according to the white balance gain. ### Claim 20 Non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: capturing, by an image sensor, an image; generating a plurality of chrominance channels by converting the image into luminance-chrominance space; performing homogeneous region segmentation on the plurality of chrominance channels to generate one or more regions of interest in the plurality of chrominance channels; and projecting the regions of interest onto eigen-illuminant images to determine gray color pixels on the image, wherein the eigen-illuminant images are generated via performing a machine learning algorithm on a training set of images captured by the image sensor.