# US 2020/0288110 A1 · Stereo-polarimetric compressed ultrafast photography (sp-cup) systems and methods

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0288110 A1
- Document ID: US20200288110A1
- Status: Published application
- Kind: A1
- Application number: 16/810,630
- Filed: 2020-03-05
- Published: 2020-09-10
- Earliest claimed: 2019-03-05
- Assignees: California Institute of Technology
- Applicants: California Institute of Technology
- Inventors: Lihong Wang; Peng Wang; Jinyang Liang; Liren Zhu
- Web page: https://patentgrep.com/patent/US20200288110A1

## CPC

- H04N 13/218: using spatial multiplexing
- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- G03B 35/00: Stereoscopic photography
- H04N 13/161: Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597)
- H04N 13/25: using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 2213/001: Constructional or mechanical details
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)

## Abstract

Systems and methods for stereo-polarimetric compressed-sensing ultrafast imaging.

## Claims

### Claim 1

A stereo-polarimetric compressed ultrafast photography (SP-CUP) system, comprising:
a dual-channel subsystem configured to receive light scattered by a subject being imaged and pass light as a first series of images and a second series of images rotated with respect to the first set of images;
a first camera configured to receive light relayed from the dual-channel generation system and record raw images of a first view and a second view;
a spatial-encoding subsystem configured to (i) receive light relayed from the dual-channel generation system, (ii) spatially encode the first series of images with a first set of complementary patterns to generate a third series of spatially-encoded images and a fourth series of spatially-encoded images, and (iii) spatially encode the second series of images with a second set of complementary patterns to generate a fifth series of spatially-encoded images and a sixth series of spatially-encoded images; and
a streak camera configured to (i) receive light relayed from the spatial-encoding subsystem, (ii) temporally shear each spatially-encoded image of the third series of spatially-encoded images and spatiotemporally integrate the temporally sheared images to record at a third view, (iii) temporally shear each spatially-encoded image of the fourth series of spatially-encoded images and spatiotemporally integrate the temporally sheared images to record a fourth view, (iv) temporally shear each spatially-encoded image of the fifth series of spatially-encoded images and spatiotemporally integrate the temporally sheared images to record a fifth view, (v) temporally shear each spatially-encoded image of the sixth series of spatially-encoded images and spatiotemporally integrate the temporally sheared image to record a sixth view.

### Claim 2 (depends on 1)

The SP-CUP system of claim 1, further comprising one or more polarizers inserted in at least one of the first view, the second view, the third view, the fourth view, the fifth view, and the sixth view.

### Claim 3 (depends on 2)

The SP-CUP system of claim 2, further comprising a computing device in electrical communication with the first camera and the streak camera to receive image data, wherein the computing device is configured to reconstruct a series of stereo-polarimetric images using image data from one or more of the first view, second view, third view, fourth view, fifth view, and sixth view.

### Claim 4 (depends on 3)

The SP-CUP system of claim 3, wherein the series of stereo-polarimetric images is at a frame rate of about 10 trillion frames per second.

### Claim 5 (depends on 2)

The SP-CUP system of claim 2, further comprising a computing device in electrical communication with the first camera and the streak camera to receive image data, wherein the computing device is configured to reconstruct a plurality of images using image data of one or more of the first view, the second view, the third view, the fourth the view, fifth view, and the sixth view, wherein the plurality of images reconstructed are one of stereo-polarimetric images, plano-polarimetric images, and planar images.

### Claim 6 (depends on 1)

The SP-CUP system of claim 1, wherein the streak camera is configured to temporally shear each spatially-encoded image by deflecting a distance according to time-of arrival.

### Claim 7 (depends on 1)

The SP-CUP system of claim 1, wherein the streak camera comprises an entrance slit that is in a fully opened position while light is received from the spatial-encoding subsystem.

### Claim 8 (depends on 1)

The SP-CUP system of claim 1, further comprising a pulsed laser configured to illuminate the subject being imaged with a single light pulse during image acquisition by the first camera and the streak camera.

### Claim 9 (depends on 1)

The SP-CUP system of claim 1, wherein the dual-channel subsystem comprises at least one optical component configured to receive light and generate a first component relayed to the first camera and a second component relayed to the spatial-encoding subsystem.

### Claim 10 (depends on 1)

The SP-CUP system of claim 1, wherein the dual-channel subsystem comprises at least one optical component configured to rotate the first view and/or the second view such that the first view is rotated by 180° in image coordinates relative to the second view.

### Claim 11 (depends on 1)

The SP-CUP system of claim 1, wherein the at least one optical component comprises a pair of dove prisms.

### Claim 12 (depends on 1)

The SP-CUP system of claim 1, wherein at least one of the first set of complimentary patterns and the second set of complimentary patterns is a pseudo-random binary spatial pattern.

### Claim 13 (depends on 1)

The SP-CUP system of claim 1, wherein the spatial-encoding subsystem comprises a digital micromirror device.

### Claim 14 (depends on 13)

The SP-CUP system of claim 13, wherein the digital micromirror device comprises an array of micromirrors, each micromirror operable to be in a first position to reflect light at a first reflection angle or in a second position to reflect light at a second reflection angle.

### Claim 15 (depends on 14)

The SP-CUP system of claim 14, wherein the spatial-encoding subsystem receives a first light beam at a first region and a second light beam at a second region from the dual-channel generation system;
wherein the third series of spatially-encoded images includes portions of the first series of images reflected by micromirrors in the first region that are in the first position;
wherein the fourth series of spatially-encoded images includes portions of the first series of images reflected by micromirrors in the first region that are the second position;
wherein the fifth series of spatially-encoded images includes portions of the second series of images reflected by micromirrors in the second region that are in the first position; and
wherein the sixth series of spatially-encoded images includes portions of the second series of images reflected by micromirrors in the second region that are the second position.

### Claim 16 (depends on 1)

The SP-CUP system of claim 1, further comprising front optics configured to collect light scattered by the subject being imaged and relay light to the dual-channel subsystem.

### Claim 17

A stereo-polarimetric compressed ultrafast photography (SP-CUP) method, comprising:
rotating a first series of images and/or a second series of images to be rotated with respect to each other;
record the first series of images of a first view and the second series of images of a second view;
spatially encoding the first series of images with a first set of complementary pseudo-random binary spatial patterns to generate a third series of spatially-encoded images and a fourth series of spatially-encoded images;
spatially encoding the second series of images with a second set of complementary pseudo-random binary spatial patterns to generate a fifth series of spatially-encoded images and a sixth series of spatially-encoded images;
temporally shearing each spatially-encoded image of the third series, spatiotemporally integrating the temporally sheared images, and recording the third view;
temporally shearing each spatially-encoded image of the fourth series, spatiotemporally integrating the temporally sheared images, and recording the fourth view;
temporally shearing each spatially-encoded image of the fifth series, spatiotemporally integrating the temporally sheared images, and recording the fifth view; and
temporally shearing each spatially-encoded image of the sixth series, spatiotemporally integrating the temporally sheared images, and recording the sixth view.

### Claim 18 (depends on 17)

The SP-CUP method of claim 17, further comprising polarizing light of at least one of the first view, second view, third view, fourth view, fifth view, and sixth view.

### Claim 19 (depends on 18)

The SP-CUP method of claim 18, further comprising reconstructing a series of stereo-polarimetric images using image data recorded from the first view, the second view, the third view, the fourth view, the fifth view, and the sixth view.

### Claim 20 (depends on 19)

The SP-CUP method of claim 19, wherein the series of stereo-polarimetric images is at a frame rate of about 10 trillion frames per second.

### Claim 21 (depends on 18)

The SP-CUP method of claim 18, wherein temporal shearing and spatiotemporally integrating is performed by a streak camera with an entrance slit in a fully opened position.
