# US 2021/0344859 A1 · Methods and systems for representing video in continuous time > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0344859 A1 - Document ID: US20210344859A1 - Status: Published application - Kind: A1 - Application number: 17/305,250 - Filed: 2021-07-02 - Published: 2021-11-04 - Earliest claimed: 2019-07-17 - Assignees: Solsona Enterprise, LLC - Applicants: Solsona Enterprise, LLC - Inventors: Chong Lee - Web page: https://patentgrep.com/patent/US20210344859A1 ## CPC - H04N 25/772: comprising A/D, V/T, V/F, I/T or I/F converters - H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract The present invention encompasses methods and systems of representing video in continuous time. Photons incident on pixels in an imaging array are continuously captured using a continuous time imaging sensor array to produce respective continuous time analog signals without any discontinuity in time. At each pixel, the respective continuous time analog signal is modulated into respective continuous time binary analog signals. The continuous time binary analog signals from all pixels are aggregated to produce a frame free video. Each pixel may comprise a continuous time sigma delta modulator and a photodiode having a capacitance. At each pixel, the respective continuous time analog signal is modulated using the continuous time sigma delta modulator to produce the corresponding continuous time binary analog signal. The capacitance of the photodiode functions as a charge integrator. ## Claims ### Claim 1 A method for representing video in continuous time, comprising: continuously capturing photons incident on pixels in an imaging array using a continuous time imaging sensor array to produce respective continuous time analog signals without any discontinuity in time; modulating, at each pixel, the respective continuous time analog signal into a respective continuous time binary analog signal; and aggregating the continuous time binary analog signals from all pixels to produce a frame free video; wherein: each pixel comprises a continuous time sigma delta modulator and a photodiode having a capacitance; at each pixel, the respective continuous time analog signal is modulated using the continuous time sigma delta modulator to produce the respective continuous time binary analog signal; and the capacitance of the photodiode functions as a charge integrator. ### Claim 2 (depends on 1) The method in accordance with claim 1, wherein the sigma delta modulator is one of charge-based or current-based. ### Claim 3 (depends on 1) The method in accordance with claim 1, further comprising: at each pixel, mapping the continuous time binary analog signal to a discrete time binary digital signal consisting of 1's and 0's. ### Claim 4 (depends on 3) The method in accordance with claim 3, further comprising: aggregating the discrete time binary digital signals from all of the pixels in the imaging array to produce a corresponding binary bit-plane per each clock cycle of the sigma delta modulator to convert the frame free video to a frame free video stream; wherein the frame free video stream comprises a time series of the binary bit-planes. ### Claim 5 (depends on 4) The method in accordance with claim 4, further comprising: converting the frame free video stream into a frame free video signal by at least one of manipulating, format converting, encoding, transcoding, compressing, storing, transmitting, and otherwise representing the frame free video stream. ### Claim 6 (depends on 3) The method in accordance with claim 3, further comprising: mapping the binary 1's and 0's from the discrete time binary digital signal to a maximum value and a minimum value, respectively, to enable continuous time video processing or display. ### Claim 7 (depends on 1) The method in accordance with claim 1, wherein: the photodiode is not reset throughout the operation of the imaging array; the charge integrator maintains a voltage range of the photodiode corresponding to a sum of a current of the photodiode and a negative of a feedback current of the sigma delta modulator. ### Claim 8 (depends on 7) The method in accordance with claim 7, further comprising: integrating the current of the photodiode continuously in time without saturating the photodiode by the charge integrator to produce an integrated value which represents total photons converted to electrical charges; each time the integrated value exceeds a threshold value, subtracting a fixed value from the integrated value to enable the charge integrator to operate within an operating range; wherein the series of subtracted values over time results in a close approximation of the total integrated value, enabling the sigma delta modulator to produce the continuous time binary analog signal. ### Claim 9 (depends on 8) The method in accordance with claim 8, wherein the charge integrator comprises: the capacitance of the photodiode; and a capacitor in parallel with the photodiode and coupled to an output of the photodiode. ### Claim 10 (depends on 8) The method in accordance with claim 8, wherein the sigma delta modulator comprises: a slicer coupled to the output of the charge integrator for determining whether the integrated value from the charge integrator output exceeds the threshold value, the slicer output comprising the discrete time binary digital signals; and a charge digital to analog converter (qDAC) coupled to an output of the slicer and to the charge integrator, which produces the fixed value to be subtracted each time the integrated value exceeds a threshold value, enabling the charge integrator to operate within an operating range. ### Claim 11 (depends on 10) The method in accordance with claim 10, wherein discrete time binary digital signals output by the slicer are mapped to the continuous time binary analog signals by the qDAC. ### Claim 12 (depends on 1) The method in accordance with claim 1, wherein the charge integrator comprises: the capacitance of the photodiode; and one or more capacitors in parallel with the photodiode and coupled to an output of the photodiode. ### Claim 13 (depends on 1) The method in accordance with claim 1, wherein the sigma delta modulator has an output which is 1-bit binary valued. ### Claim 14 (depends on 1) The method in accordance with claim 1, wherein the sigma delta modulator has an output which is more than 1-bit binary valued. ### Claim 15 A system for representing video in continuous time, comprising: a continuous time imaging sensor array for continuously capturing photons incident on pixels of the imaging array to produce respective continuous time analog signals without any discontinuity in time; each of the pixels comprising a photodiode and a sigma delta modulator circuit for modulating the respective continuous time analog signal into a respective continuous time binary analog signal; a processor for aggregating the continuous time binary analog signals from all the pixels to produce a frame free video stream; and a continuous time display array for displaying the frame free video stream; wherein: the photodiode has a capacitance; the capacitance of the photodiode functions as a charge integrator; and the modulator circuit comprises a continuous time sigma delta modulator. ### Claim 16 A pixel for a continuous time imaging sensor array, comprising: a photodiode for capturing incident photons to produce a continuous time analog signal without any discontinuity in time; a sigma delta modulator circuit for modulating the continuous time analog signal into a continuous time binary analog signal, wherein: the photodiode has a capacitance; the capacitance of the photodiode functions as a charge integrator; and the continuous time binary analog signals from all pixels in an imaging sensor array are aggregated to produce a frame free video stream.