# US 2014/0192235 A1 · Systems, methods, and media for reconstructing a space-time volume from a coded image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0192235 A1 - Document ID: US20140192235A1 - Status: Published application - Kind: A1 - Application number: 14/001,139 - Filed: 2012-02-27 - Published: 2014-07-10 - Earliest claimed: 2011-02-25 - Assignees: Sony Corporation; The Trustees of Columbia University in the City of New York - Inventors: Yasunobu Hitomi; Jinwei Gu; Mohit Gupta; Tomoo Mitsunaga; Shree K. Nayar - Web page: https://patentgrep.com/patent/US20140192235A1 ## CPC - H04N 13/106: Processing image signals (for multi-view video sequence encoding H04N19/597) - G06T 1/0007: {Image acquisition} - G06T 5/70: Denoising; Smoothing - G06T 7/50: Depth or shape recovery - H04N 13/128: Adjusting depth or disparity - H04N 13/296: Synchronisation thereof; Control thereof - H04N 13/388: Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume - H04N 23/73: by influencing the exposure time - H04N 25/40: Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled - H04N 25/535: by dynamic region selection - G06T 2200/21: involving computational photography - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems, methods, and media for reconstructing a space-time volume from a coded image are provided. In accordance with some embodiments, systems for reconstructing a space-time volume from a coded image are provided, the systems comprising: an image sensor that outputs image data; and at least one processor that: causes a projection of the space-time volume to be captured in a single image of the image data in accordance with a coded shutter function; receives the image data; and performs a reconstruction process on the image data to provide a space-time volume corresponding to the image data. ## Claims ### Claim 1 A system for reconstructing a space-time volume from a coded image, comprising: an image sensor that outputs image data; and at least one processor that: causes a projection of the space-time volume to be captured in a single image of the image data in accordance with a coded shutter function; receives the image data; and performs a reconstruction process on the image data to provide a space-time volume corresponding to the image data. ### Claim 2 (depends on 1) The system of claim 1, wherein the reconstruction process is based on an over-complete dictionary. ### Claim 3 (depends on 2) The system of claim 2, wherein the over-complete dictionary is based on rotated video samples. ### Claim 4 (depends on 1) The system of claim 1, wherein the coded shutter function has random start times for pixel exposure bumps. ### Claim 5 (depends on 1) The system of claim 1, wherein the coded shutter function has only a single exposure period per pixel per sensor integration time. ### Claim 6 (depends on 1) The system of claim 1, wherein the coded shutter function has at least one pixel in each pixel neighborhood exposed during each interval of a sensor integration time. ### Claim 7 (depends on 1) The system of claim 1, further comprising a Liquid Crystal on Silicon chip that modulates light onto the image sensor according to the coded shutter function in response to signals from the at least one processor. ### Claim 8 (depends on 1) The system of claim 1, wherein the image sensor has pixels that individually addressable. ### Claim 9 (depends on 1) The system of claim 1, wherein the reconstruction process includes performing an orthogonal matching pursuit algorithm. ### Claim 10 A method for reconstructing a space-time volume from a coded image, comprising: causing a projection of the space-time volume to be captured by an image sensor in a single image of image data in accordance with a coded shutter function using a hardware processor; receiving the image data using a hardware processor; and performing a reconstruction process on the image data to provide a space-time volume corresponding to the image data using a hardware processor. ### Claim 11 (depends on 10) The method of claim 10, wherein the reconstruction process is based on an over-complete dictionary. ### Claim 12 (depends on 11) The method of claim 11, wherein the over-complete dictionary is based on rotated video samples. ### Claim 13 (depends on 10) The method of claim 10, wherein the coded shutter function has random start times for pixel exposure bumps. ### Claim 14 (depends on 10) The method of claim 10, wherein the coded shutter function has only a single exposure period per pixel per sensor integration time. ### Claim 15 (depends on 10) The method of claim 10, wherein the coded shutter function has at least one pixel in each pixel neighborhood exposed during each interval of a sensor integration time. ### Claim 16 (depends on 10) The method of claim 10, further comprising modulating light onto the image sensor using an Liquid Crystal on Silicon chip according to the coded shutter function in response to signals from the hardware processor. ### Claim 17 (depends on 10) The method of claim 10, wherein the image sensor has pixels that are individually addressable. ### Claim 18 (depends on 10) The method of claim 10, wherein the reconstruction process includes performing an orthogonal matching pursuit algorithm. ### Claim 19 A non-transitory computer-readable medium containing computer-executable instructions that, when executed by a processor, cause the processor to perform a method for reconstructing a space-time volume from a coded image, the method comprising: causing a projection of the space-time volume to be captured in a single image of image data in accordance with a coded shutter function; receiving the image data; and performing a reconstruction process on the image data to provide a space-time volume corresponding to the image data. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the reconstruction process is based on an over-complete dictionary. ### Claim 21 (depends on 20) The non-transitory computer-readable medium of claim 20, wherein the over-complete dictionary is based on rotated video samples. ### Claim 22 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the coded shutter function has random start times for pixel exposure bumps. ### Claim 23 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the coded shutter function has only a single exposure period per pixel per sensor integration time. ### Claim 24 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the coded shutter function has at least one pixel in each pixel neighborhood exposed during each interval of a sensor integration time. ### Claim 25 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the method further comprises modulating light onto the image sensor according to the coded shutter function. ### Claim 26 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the method further comprises individually addressing pixels of an image sensor. ### Claim 27 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the reconstruction process includes performing an orthogonal matching pursuit algorithm.