# US 2008/0187305 A1 · 4D light field cameras

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2008/0187305 A1
- Document ID: US20080187305A1
- Status: Published application
- Kind: A1
- Application number: 11/671,701
- Filed: 2007-02-06
- Published: 2008-08-07
- Earliest claimed: 2007-02-06
- Assignees: Mitsubishi Electric Research Laboratories, Inc.
- Inventors: Ramesh Raskar; Amit Kumar Agrawal
- Web page: https://patentgrep.com/patent/US20080187305A1

## CPC

- G02B 27/0075: {with means for altering, e.g. increasing, the depth of field or depth of focus}
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS

## Abstract

A camera acquires a 4D light field of a scene. The camera includes a lens and sensor. A mask is arranged in a straight optical path between the lens and the sensor. The mask including an attenuation pattern to spatially modulate the 4D light field acquired of the scene by the sensor. The pattern has a low spatial frequency when the mask is arranged near the lens, and a high spatial frequency when the mask is arranged near the sensor.

## Claims

### Claim 1

An apparatus for acquiring a 4D light field of a scene, comprising:
a lens;
a sensor; and
a mask arranged in a straight optical path between the lens and the sensor, the mask including an attenuation pattern to spatially modulate the 4D light field acquired of the scene by the sensor.

### Claim 2 (depends on 1)

The apparatus of claim 1, in which the pattern has a low spatial frequency and is arranged near the lens.

### Claim 3 (depends on 1)

The apparatus of claim 1, in which the pattern has a high spatial frequency and is arranged near the sensor.

### Claim 4 (depends on 3)

The apparatus of claim 3, in which the mask heterodynes the 4D light field.

### Claim 5 (depends on 1)

The apparatus of claim 1, in which the sensor produces an input image, and further comprising:
means for reconstructing the 4D light field from the input image.

### Claim 6 (depends on 5)

The apparatus of claim 5, in which the 4D light field is reconstructed by Fourier-domain decoding.

### Claim 7 (depends on 2)

The apparatus of claim 2, in which the sensor produces a defocused blur input image.

### Claim 8 (depends on 2)

The apparatus of claim 2, in which digital refocusing can be achieved without a loss of resolution. The resolution of the refocused image is equal to the pixel resolution of the sensor.

### Claim 9 (depends on 1)

The apparatus of claim 1, in which the pattern in printed on the mask.

### Claim 10 (depends on 1)

The apparatus of claim 1, in which the mask is active, and further comprising:
means for changing a position of the mask and the spatially frequency of the pattern continuously.

### Claim 11 (depends on 5)

The apparatus of claim 5, further comprising:
means for refocusing the input image from the reconstructing 4D light field to produce an in-focus output image, without a loss of resolution.

### Claim 12 (depends on 11)

The apparatus of claim 11, in which the output image has an increased depth of field.

### Claim 13 (depends on 5)

The apparatus of claim 5, further comprising:
means for synthesizing a novel refocused image from the reconstructed 4D light field.

### Claim 14 (depends on 5)

The apparatus of claim 5, further comprising:
means for generating a high resolution image from the reconstructed 4D light field.

### Claim 15 (depends on 1)

The apparatus of claim 1, in which the pattern includes colors.

### Claim 16 (depends on 1)

The apparatus of claim 1, in which the mask is placed at an angle with respect to the optical path.

### Claim 17 (depends on 1)

The apparatus of claim 1, in which the mask has a cosine pattern.

### Claim 18 (depends on 1)

The apparatus of claim 1, in which the mask uses polarizing effects.

### Claim 19 (depends on 1)

The apparatus of claim 1, in which the pattern has a high spatial frequency and is arranged near the lens to precisely sense depth in the scene.

### Claim 20 (depends on 7)

The apparatus of claim 7, further comprising:
means for deblurring the input image to remove out of focus blur.

### Claim 21 (depends on 1)

The apparatus of claim 1, in which the sensor produces an input image encoding the 4D light field, and further comprising:
a microprocessor connected to the sensor for generating an output image from the input image.

### Claim 22 (depends on 1)

The apparatus of claim 1, in which the sensor acquires a sequence of images of the scene.

### Claim 23

A method for acquiring a 4D light field of a scene, comprising the steps of:
passing a 4D light field through a lens to a sensor along a straight optical path; and
modulating the 4D light field with an attenuating mask arranged in the optical path between the lens and the sensor.

### Claim 24 (depends on 23)

The method of claim 23, in which the pattern has a low spatial frequency and is arranged near the lens.

### Claim 25 (depends on 23)

The method of claim 23, in which the pattern has a high spatial frequency and is arranged near the sensor.

### Claim 26 (depends on 25)

The method of claim 25, in which the mask heterodynes the 4D light field.

### Claim 27 (depends on 23)

The method of claim 23, in which the sensor produces an input image, and further comprising the step of:
reconstructing the 4D light field from the input image.

### Claim 28 (depends on 27)

The method of claim 27, in which the 4D light field is reconstructed by Fourier-domain decoding.

### Claim 29 (depends on 24)

The method of claim 24, in which the sensor produces a defocused blur input image.

### Claim 30 (depends on 29)

The method of claim 29, in which the refocused image obtained by deblurring the input image has same resolution as the pixel resolution of the sensor.

### Claim 31 (depends on 23)

The method of claim 23, in which the pattern in printed on the mask.

### Claim 32 (depends on 23)

The method of claim 23, in which the mask is active, and further comprising the step:
changing a position of the mask and the spatially frequency of the pattern continuously.

### Claim 33 (depends on 5)

The apparatus of claim 5, further comprising the step of:
refocusing the input image from the reconstructed 4D light field to produce an in-focus output image.

### Claim 34 (depends on 33)

The method of claim 33, in which the output image has an increased depth of field.

### Claim 35 (depends on 27)

The method of claim 27, further comprising the step of:
synthesizing a novel image from the reconstructed 4D light field.

### Claim 36 (depends on 27)

The method of claim 27, further comprising the step of:
generating a high resolution image from the reconstructed 4D light field.

### Claim 37 (depends on 23)

The method of claim 23, in which the pattern includes colors.

### Claim 38 (depends on 23)

The method of claim 23, in which the mask is placed at an angle with respect to the optical path.

### Claim 39 (depends on 23)

The method of claim 23, in which the mask has a cosine pattern.

### Claim 40 (depends on 23)

The method of claim 23, in which the mask uses polarizing effects.

### Claim 41 (depends on 23)

The method of claim 23, in which the pattern has a high spatial frequency and is arranged near the lens to precisely sense depth in the scene.

### Claim 42 (depends on 29)

The method of claim 29, further comprising the step of:
deburring the input image.

### Claim 43 (depends on 23)

The method of claim 23, in which the sensor produces an input image encoding the 4D light field, and further comprising the step of:
generating an output image from an input image produced by the sensor.

### Claim 44 (depends on 23)

The method of claim 23, in which the sensor acquires a sequence of images of the scene.

### Claim 45 (depends on 23)

The method of claim 23, further comprising the steps of:
applying a Fourier transform to a 2D signal sensed by the sensor to obtain 2D tiles;
re-assembling the 2D tiles into a 4D stack of planes; and
applying an inverse Fourier transform to the 4D stack of tiles to reconstruct the 4D light field.

### Claim 46 (depends on 23)

The method of claim 23, in which the mask is arranged in an aperture, and in which a spectrum of the 4D light field of the scene lacks variation in an angular dimensions, and the mask is arranged in an aperture and a spectrum of the mask lacks variation in the spatial dimensions and the sensor senses a convolution of the two spectrums.

### Claim 47 (depends on 45)

The method of claim 45, in which the Fourier transform of the 4D light field l after passing through the aperture is given by a modulation of the Fourier transform of the incoming light field with the Fourier transform of the aperture modulation function.

### Claim 48 (depends on 27)

The method of claim 27, further comprising:
determining a re focused image at a different depth from the input image.

### Claim 49 (depends on 27)

The method of claim 27, further comprising:
synthesizing a novel image from a different viewpoint from the reconstructed 4D light field.

### Claim 50 (depends on 23)

The method of claim 23, in which multiple masks are arranged in the optical path.

### Claim 52 (depends on 23)

The method of claim 23, in which the pattern is an analytical 2D function.

### Claim 53 (depends on 23)

The method of claim 23, in which the pattern is a binary function.

### Claim 54 (depends on 23)

The method of claim 23, in which the pattern is a continuous 2D function.

### Claim 55 (depends on 50)

The method of claim 50, in which the sensor acquires a sequence of images of the scene with the multiple masks.

### Claim 56

A method for acquiring a 4D light field of a scene, comprising the steps of:
modulating a 4D light field with an attenuating mask arranged in a straight optical path between a lens and a sensor; and
sensing the modulated 4D light field by the sensor to produce an input image encoding the 4D light field.

### Claim 57 (depends on 56)

The method of claim 56, further comprising:
demodulating the input image to recover the 4D light field.
