# US 10,771,722 B2 · Methods for enabling in-field selection of near-sensor digital imaging functions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,771,722 B2
- Document ID: US10771722B2
- Status: Granted
- Kind: B2
- Application number: 16/145,272
- Filed: 2018-09-28
- Published: 2019-04-04
- Granted: 2020-09-08
- Earliest claimed: 2017-09-29
- Assignees: Massachusetts Institute of Technology
- Applicants: Peter J. Grossmann; Matthew Stamplis; Kate Thurmer; Brian Tyrrell; Jonathan Frechette
- Inventors: Peter J. Grossmann; Matthew Stamplis; Kate Thurmer; Brian Tyrrell; Jonathan Frechette
- Web page: https://patentgrep.com/patent/US10771722B2

## CPC

- H04N 25/76: Addressed sensors, e.g. MOS or CMOS sensors
- H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
- H04N 25/79: Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors
- H10F 39/184: {Infrared image sensors}
- H10F 39/809: {of hybrid image sensors}
- H03K 19/17744: {for input/output signals}
- H03M 9/00: Parallel/series conversion or vice versa (digital stores in which the information is moved stepwise per se G11C19/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION
- H03K: PULSE TECHNIQUE (measuring pulse characteristics G01R; modulating sinusoidal oscillations with pulses H03C; transmission of digital information H04L; discriminator circuits detecting phase difference between two signals by counting or integrating cycles of oscillation H03D3/04; automatic control, starting, synchronisation or stabilisation of generators of electronic oscillations or pulses where the type of generator is irrelevant or unspecified H03L; coding, decoding or code conversion, in general H03M)
- H03M: CODING; DECODING; CODE CONVERSION IN GENERAL (using fluidic means F15C4/00; optical analogue/digital converters G02F7/00; coding, decoding or code conversion, specially adapted for particular applications, see the relevant subclasses, e.g. G01D, G01R, G06F, G06T, G09G, G10L, G11B, G11C, H04B, H04L, H04M, H04N; ciphering or deciphering for cryptography or other purposes involving the need for secrecy G09C)

## Abstract

An imaging device is often paired with a readout integrated circuit (ROIC), which provides processing and data transfer functionality. The circuitry of a ROIC is typically specialized to meet the requirements of an application, which limits the ROIC to a few modes of operation and restricts compatibility to only certain types of imaging devices and applications. Furthermore, the circuitry supporting the processing functionality is limited due to size constraints on the ROIC. These shortcomings can be overcome with a field programmable imaging array (FPIA), which can be implemented as an integrated circuit combining customized ROIC sensor interface circuitry with field programmable gate array (FPGA) circuitry to enable post-fabrication definition of ROIC operational modes. An FPIA chip may form part of a three-chip stack that also includes an analog sensor interface chip for analog-to-digital conversion and an imaging device.

## Claims

### Claim 1

A field programmable imaging array (FPIA) chip, comprising:
a substrate;
a plurality of macropixel elements, disposed on the substrate, to process digital data, each one of the macropixel elements including non-reconfigurable circuitry;
a plurality of field programmable gate array (FPGA) elements, disposed on at least a portion of the substrate in electronic communication with the plurality of macropixel elements, to receive processed digital data from the plurality of macropixel elements;
a plurality of reconfigurable interconnects, disposed on the substrate, to reconfigurably connect the plurality of macropixel elements to the plurality of FPGA elements; and
a plurality of deserializer elements, disposed on the substrate, to convert serial digital data from the plurality of macropixel elements to parallel digital data for processing by the plurality of FPGA elements.

### Claim 2 (depends on 1)

The FPIA chip of claim 1, wherein a first reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to each macropixel element.

### Claim 3 (depends on 1)

The FPIA chip of claim 1, wherein a second reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one macropixel element in the plurality of macropixel elements to a first deserializer element in the plurality of deserializer elements.

### Claim 4 (depends on 3)

The FPIA chip of claim 3, wherein a third reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element in the plurality of FPGA elements to the first deserializer element.

### Claim 5 (depends on 1)

The FPIA chip of claim 1, further comprising:
a plurality of input/output (I/O) elements, disposed on the substrate, to at least one of receive a control signal or supply a signal output.

### Claim 6 (depends on 5)

The FPIA chip of claim 5, wherein a fourth reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first I/O element in the plurality of I/O elements.

### Claim 7

A method of processing data with a field programmable imaging array (FPIA) chip, the FPIA chip comprising a substrate, macropixel elements disposed on the substrate, and field programmable gate array (FPGA) elements disposed on the substrate in electronic communication with the plurality of FPGA elements, the method comprising:
processing digital data with the macropixel elements;
transferring processed digital data from the macropixel elements to the FPGA elements;
reconfiguring connections between the macropixel elements and the FPGA elements; and
converting serial digital data from the macropixel elements to parallel digital data for processing by the FPGA elements.

### Claim 8 (depends on 7)

The method of claim 7, wherein reconfiguring connections between the macropixel elements and the FPGA elements comprises:
connecting a first macropixel element to a first FPGA element at a first time step; and
connecting the first macropixel element to a second FPGA element at a second time step.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
reconfigurably coupling a first macropixel element to a deserializer element that converts the serial digital data to parallel digital data.

### Claim 10 (depends on 9)

The method of claim 9, further comprising
reconfigurably connecting the deserializer element to a first FPGA element.

### Claim 11 (depends on 7)

The method of claim 7, further comprising:
reconfigurably connecting a first FPGA element to an input/output (I/O) element; and
receiving a control signal and/or supplying a signal output with the I/O element.

### Claim 12

An apparatus, comprising:
a sensor array chip to generate analog data;
a sensor interface chip, having a first side disposed on a first side of the sensor array chip, to convert the analog data to digital data; and
a digital processing chip, having a first side disposed on a second side of the sensor interface chip, to process the digital data provided by the sensor interface chip, the digital processing chip comprising a plurality of deserializer elements to convert serial digital data to parallel digital data,
wherein a second reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one macropixel element to a first deserializer element in the plurality of deserializer elements.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the digital processing chip comprises a field programmable imaging array (FPIA) chip, the FPIA chip comprising:
a plurality of macropixel elements to receive and/or generate the digital data from the sensor interface chip inputs and outputs; and
a plurality of field programmable gate array (FPGA) elements, in electronic communication with the plurality of macropixel elements, to process the digital data.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein a first macropixel element in the plurality of macropixel elements is configured to receive digital data representing light detected by a first sensor in the sensor array chip.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the first macropixel is configured to be dynamically reconfigured to receive digital data representing light detected by a second sensor in the sensor array chip.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the first macropixel element includes non-reconfigurable circuitry.

### Claim 17 (depends on 13)

The apparatus of claim 13, further comprising:
a plurality of reconfigurable interconnects, disposed on the FPIA chip, to reconfigurably connect the plurality of macropixel elements to the plurality of FPGA elements.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein a first reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first macropixel element in the plurality of macropixel elements.

### Claim 19 (depends on 17)

The apparatus of claim 17, further comprising:
a plurality of input/output (I/O) elements, disposed on the FPIA chip, to receive a control signal and provide a signal output.

### Claim 20 (depends on 19)

The apparatus of claim 19, wherein a fourth reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to a first I/O element in the plurality of I/O elements.

### Claim 21 (depends on 12)

The apparatus of claim 12, wherein a third reconfigurable interconnect in the plurality of reconfigurable interconnects is configured to be dynamically reconfigured to electronically couple at least one FPGA element to the first deserializer element.