# US 11,910,114 B2 · Multi-mode image sensor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,910,114 B2 - Document ID: US11910114B2 - Status: Granted - Kind: B2 - Application number: 17/364,763 - Filed: 2021-06-30 - Published: 2022-01-20 - Granted: 2024-02-20 - Earliest claimed: 2020-07-17 - Assignees: Meta Platforms Technologies, LLC - Applicants: Meta Platforms Technologies, LLC - Inventors: Andrew Samuel Berkovich - Web page: https://patentgrep.com/patent/US11910114B2 ## CPC - H04N 25/443: by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming - H04N 25/709: Circuitry for control of the power supply - H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof - H04N 25/40: Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled - H04N 25/707: Pixels for event detection - H04N 25/772: comprising A/D, V/T, V/F, I/T or I/F converters - H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters - H04N 25/79: Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract In some examples, an apparatus comprises: an array of pixel cells, each pixel cell of the array of pixel cells configured to output a voltage; first analog-to-digital converters (ADCs), each first ADC being associated with a pixel cell or a block of pixel cells; second ADCs, each second ADC being associated with a column of pixel cells of the array of pixel cells; and a controller configured to: in a first mode, enable at least a subset of the first ADCs to perform a quantization operation of the voltages output by at least a subset of the pixel cells in parallel to generate a first image frame; and in a second mode, enable at least a subset of the second ADCs to perform quantization operations sequentially of the voltages output by pixel cells within at least a subset of the columns of pixel cells to generate a second image frame. ## Claims ### Claim 1 An apparatus comprising: an array of pixel cells, each pixel cell of the array of pixel cells including one or more photodiodes configured to generate charge in response to light, and a charge storage device to convert the charge to output a voltage; first analog-to-digital converters (ADCs), each first ADC being associated with a pixel cell or a block of pixel cells of the array of pixel cells; second ADCs, each second ADC being associated with a column of pixel cells or a column of blocks of pixel cells of the array of pixel cells; and a controller configured to: in a first mode, enable at least a subset of the first ADCs to perform a quantization operation of the voltages output by at least a subset of the pixel cells or the blocks of pixel cells associated with the at least the subset of the first ADCs in parallel to generate a first image frame; in a second mode, enable at least a subset of the second ADCs to perform quantization operations sequentially of the voltages output by pixel cells or blocks of pixel cells within at least a subset of the columns of pixel cells or the columns of blocks of pixel cells associated with the at least the subset of the second ADCs to generate a second image frame; and wherein the controller is configured to switch between the first mode and the second mode based on one or more operation conditions, the one or more operation conditions comprising at least one of: a sparsity of an image frame to be generated, a frame period in which the image frame is to be generated, or a constraint for resources to be used in generating and transmitting the image frame. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the controller is configured to enable a subset of the array of pixel cells to generate voltages, and to enable a subset of the first ADCs or the second ADCs to generate the first image frame or the second image frame as a sparse image frame. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the array of pixel cells are formed in a first semiconductor layer; wherein the first ADCs are formed in a second semiconductor layer; wherein the first semiconductor layer and the second semiconductor layer form a stack; and wherein the first ADCs are positioned directly below the array of pixel cells. ### Claim 4 (depends on 3) The apparatus of claim 3, wherein the first ADCs are connected to the array of pixel cells via vertical interconnects. ### Claim 5 (depends on 4) The apparatus of claim 4, wherein the second ADCs are formed in the second semiconductor layer and on a periphery of the first ADCs. ### Claim 6 (depends on 1) The apparatus of claim 1, further comprising: column signal lines, each column signal line being connected to a second ADC of the second ADCs and associated with a column of pixel cells or a column of blocks of pixel cells; and row selection switches, wherein each row selection switch connects between a column signal line and a pixel cell of the column of pixel cells associated with the column signal line. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein the controller is configured to, in the second mode, transmit row selection signals to the row selection switches to select one pixel cell from each of the at least a subset of the columns of pixel cells to access one second ADC of the at least a subset of the second ADCs. ### Claim 8 (depends on 7) The apparatus of claim 7, wherein the controller is configured to, in the second mode, transmit the row selection signals to the row selection switches to select one row of pixel cells from the at least the subset of the columns of pixel cells to access the at least a subset of the second ADCs in parallel. ### Claim 9 (depends on 6) The apparatus of claim 6, further comprising column selection switches forming pairs of serial switches with row selection switches; and wherein a pair of a row selection switch and a column selection switch connects between a column signal line and a pixel cell of a block of pixel cells. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the controller is configured to: in the first mode, transmit a first sequence of row selection signals and column selection signals to the pairs of serial switches to enable each pixel cell within a block of pixel cells to sequentially access the first ADC associated with the block of pixel cells; and in the second mode, transmit a second sequence of row selection signals and column selection signals to the pairs of serial switches to enable pixel cells within the at least the subset of the columns of blocks of pixel cells to sequentially access the second ADCs associated with the subset of the columns of blocks of pixel cells. ### Claim 11 (depends on 9) The apparatus of claim 9, further comprising: first switches controllable by the controller to connect the first ADCs to the pairs of serial switches in the first mode and to disconnect the first ADCs from the pairs of serial switches in the second mode; and second switches controllable by the controller to connect the second ADCs to the pairs of serial switches in the second mode and to disconnect the second ADCs from the pairs of serial switches in the first mode. ### Claim 12 (depends on 6) The apparatus of claim 6, further comprising analog pre-processing circuits, each analog pre-processing circuit being interposed between a column signal line and a second ADC. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein each analog pre-processing circuit is configured to perform at least one of: a summation or filtering of analog outputs received from the column signal line, or a detection of changes in the analog outputs received from the column signal line between image frames. ### Claim 14 (depends on 13) The apparatus of claim 13, wherein each analog pre-processing circuit is configured to transmit a start signal to the second ADC to start a quantization operation of the analog outputs based on the detected changes exceeding a threshold. ### Claim 15 (depends on 1) The apparatus of claim 1, wherein the controller is configured to operate in the second mode based on the sparsity of the image frame exceeding a threshold. ### Claim 16 (depends on 1) The apparatus of claim 1, wherein the controller is configured to operate in the second mode such that a power consumption of the apparatus is within the constraint. ### Claim 17 (depends on 1) The apparatus of claim 1, wherein the controller is configured to operate in the first mode based on a frame period being below a threshold. ### Claim 18 A method comprising: operating each pixel cell of at least a subset of an array of pixel cells to generate charge in response to light, and to convert the charge to output a voltage; in a first mode, enabling at least a subset of first ADCs to perform a quantization operation of the voltages output by at least the subset of the pixel cells or blocks of pixel cells in parallel to generate a first image frame, each first ADC being associated with a pixel cell or a block of pixel cells of the array of pixel cells; in a second mode, enabling at least a subset of second ADCs to perform quantization operations sequentially for voltages output by pixel cells or blocks of pixel cells within at least a subset of columns of pixel cells or columns of blocks of pixel cells to generate a second image frame, each second ADC being associated with a column of pixel cells or a column of blocks of pixel cells of the array of pixel cells; and switching between the first mode and the second mode based on one or more operation conditions, the one or more operation conditions comprising at least one of: a sparsity of an image frame to be generated, a frame period in which the image frame is to be generated, or a constraint for resources to be used in generating and transmitting the image frame.