# US 12,225,315 B2 · Dynamic pixel-wise multi-gain readout for high dynamic range imaging > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,225,315 B2 - Document ID: US12225315B2 - Status: Granted - Kind: B2 - Application number: 18/045,639 - Filed: 2022-10-11 - Published: 2023-04-13 - Granted: 2025-02-11 - Earliest claimed: 2021-10-11 - Assignees: Gigajot Technology, Inc. - Applicants: Gigajot Technology, Inc. - Inventors: Jiaju Ma - Web page: https://patentgrep.com/patent/US12225315B2 ## CPC - H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters - H04N 23/76: by influencing the image signals - H04N 25/11: Arrangement of colour filter arrays [CFA]; Filter mosaics - H04N 25/51: Control of the gain - H04N 25/59: by controlling the amount of charge storable in the pixel, e.g. modification of the charge conversion ratio of the floating node capacitance - H04N 25/778: comprising amplifiers shared between a plurality of pixels, i.e. at least one part of the amplifier must be on the sensor array itself - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems and methods are disclosed to enable fast readout from an image sensor to support pixel-wise conversion gain selection for high dynamic range imaging. In embodiments, an image sensor integrated circuit performs the pixel-wise gain selection with its readout circuitry, so that the image sensor outputs pixels with only the selected gain option. In this manner, the image sensor is able to achieve faster frame rates and lower power consumption. Depending on the embodiment, the conversion gain may be selected by the readout logic, an image signal processor, or an auto-exposure engine. The gain selection may be made based on a previous image captured by the camera or other pixels in the same image. The image signal processor may interpolate a high-gain and a low-gain portion of the image to obtain full resolution images in the two gain options, and merge the two to obtain the final image. ## Claims ### Claim 1 An image sensor integrated circuit, comprising: a plurality of photodiodes that implement pixels in a pixel array, configured to capture a frame; readout logic configured to: receive a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels, including a high conversion gain (HCG) and a low conversion gain (LCG); read a first pixel value of the frame from a first pixel with the HCG as indicated by the gain selection signal; read, after the read of the first pixel, a second pixel value of the frame from a second pixel with LCG as indicated by the gain selection signal, wherein the gain selection signal is changed after the read of the first pixel; and output the pixel values of the frame via an output interface of the image sensor integrated circuit, wherein an individual pixel value is output for one conversion gain. ### Claim 2 (depends on 1) The system of claim 1, wherein the gain selection signal is received from another device distinct from the image sensor integrated circuit. ### Claim 3 (depends on 1) The system of claim 1, wherein: the pixels organized as a plurality of pixel units individually associated with shared readout circuitry; and the gain selection signal specifies one of multiple pixel gain configuration patterns for the pixel units. ### Claim 4 (depends on 3) The system of claim 3, wherein: an individual pixel unit is a 2×2 group of four pixels; and the pixel gain configuration patterns include three or more of: a high gain only pattern that specifies HCG for all four pixels, a low-light focused pattern that specifies HCG for three of the four pixels, a balanced pattern that specifies HCG for two of the four pixels, a high-light focused pattern that specifies HCG for one of the four pixels, and a low-gain only pattern that specifies LCG for all four pixels. ### Claim 5 (depends on 3) The system of claim 3, further comprising: a color filter array (CFA) configured to filter light received by the pixels for different colors, wherein the CFA uses one color for pixels in a same pixel unit. ### Claim 6 (depends on 1) The system of claim 1, wherein the readout logic is configured to: read one or more initial pixels in a local region of the pixel array; and determine one or more conversion gains for one or more later pixels in the local region based at least in part on one or more pixel values of the one or more initial pixels. ### Claim 7 (depends on 6) The system of claim 6, wherein the one or more conversion gains for the one or more later pixels are determined based at least in part on a particular pixel value in the local region exceeding a threshold. ### Claim 8 (depends on 6) The system of claim 6, wherein: the one or more conversion gains for the one or more later pixels are determined by a regional gain control unit dedicated to a subset of the pixels in the local region; the readout logic includes a plurality of regional gain control units dedicated to different subsets of the pixels; and the regional gain control units are implemented on an application-specific integrated circuit (ASIC) layer coupled to a pixel layer that implements the pixel array. ### Claim 9 A camera, comprising: one or more lenses configured to receive light; an image sensor comprising: a plurality of photodiodes that implement pixels in a pixel array, wherein the pixels are configured to capture a frame according to the light; readout logic configured to: receive a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels, including a high conversion gain (HCG) and a low conversion gain (LCG); read a first pixel value of the frame from a first pixel with the HCG as indicated by the gain selection signal; read, after the read of the first pixel, a second pixel value of the frame from a second pixel with LCG as indicated by the gain selection signal, wherein the gain selection signal is changed after the read of the first pixel; and output the pixel values of the frame via an output interface, wherein an individual pixel value is output for one conversion gain; and an image signal processor (ISP) configured to: receive the pixel values output by the image sensor; and process the pixel values to construct an image associated with the frame. ### Claim 10 (depends on 9) The camera of claim 9, wherein the gain selection signal is generated by the ISP based at least in part on a previous image captured by the camera. ### Claim 11 (depends on 9) The camera of claim 9, wherein the gain selection signal is generated by an auto-exposure engine implemented by the camera. ### Claim 12 (depends on 9) The camera of claim 9, wherein: the pixels are organized as a plurality of pixel units individually associated with shared readout circuitry; and the gain selection signal specifies one of multiple pixel gain configuration patterns for individual ones of the pixel units. ### Claim 13 (depends on 9) The camera of claim 9, wherein: the image sensor is implemented on a die; and the ISP is implemented on the same die. ### Claim 14 (depends on 9) The camera of claim 9, wherein the ISP is configured to: construct a HGC portion of the image with HCG pixel values received from the image sensor and a LCG portion of the image with LCG pixel values received from the image sensor; interpolate the HGC and LCG portions to estimate missing pixel values in the HGC and LCG portions; perform gain calibration on the LCG portion to reduce gain differences between the HGC and LCG portions; and merge the HGC and LCG portions to construct a final image. ### Claim 15 (depends on 14) The camera of claim 14, wherein the ISP is configured to interpolate the HCG and LCG portions of the image using one or more of: a bi-linear algorithm, a nearest neighbor algorithm, a bi-cubic algorithm, and a neural network-based algorithm. ### Claim 16 A method, comprising: capturing, responsive to light, a frame via a plurality of photodiodes that implement pixels in a pixel array of an image sensor; receiving a gain selection signal indicating respective conversion gains for reading pixel values of the frame from individual ones of the pixels, including a high conversion gain (HCG) and a low conversion gain (LCG); reading, via readout logic of the image sensor, a first pixel value of the frame from a first pixel with the HCG as indicated by the gain selection signal; reading, after the reading of the first pixel, a second pixel value of the frame from a second pixel with LCG as indicated by the gain selection signal, wherein the gain selection signal is changed after the read of the first pixel; and outputting the pixel values of the frame via an output interface of the image sensor, wherein an individual pixel value is output for one conversion gain. ### Claim 17 (depends on 16) The method of claim 16, wherein: the pixels are read as a plurality of pixel units via respective readout circuitry associated with the pixel units; and the gain selection signal specifies one of multiple pixel gain configuration patterns for individual ones of the pixel units. ### Claim 18 (depends on 16) The method of claim 16, wherein the gain selection signal is generated by the readout logic. ### Claim 19 (depends on 16) The method of claim 16, further comprising processing, via an image signal processor (ISP), the pixel values to construct an image, wherein the gain selection signal is generated by the ISP. ### Claim 20 (depends on 19) The method of claim 19, wherein the processing of the pixel values comprises: interpolating a HGC portion of the image with HCG pixel values output by the image sensor and a LCG portion of the image with LCG pixel values output by the sensor to estimate missing pixel values in the HCG and LCG portions; and merging the HGC and LCG portions to construct the image.