# US 2012/0249819 A1 · Multi-modal image capture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2012/0249819 A1
- Document ID: US20120249819A1
- Status: Published application
- Kind: A1
- Application number: 13/073,634
- Filed: 2011-03-28
- Published: 2012-10-04
- Earliest claimed: 2011-03-28
- Assignees: Canon Kabushiki Kaisha
- Inventors: Francisco Imai
- Web page: https://patentgrep.com/patent/US20120249819A1

## CPC

- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/63: by using electronic viewfinders
- H04N 23/675: {comprising setting of focusing regions}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

An image capture apparatus includes an image sensor which has a tunable spectral response and which is tunable in accordance with a capture mask to capture multi-spectral image data. Light-field optics are arranged to project light-field information of a scene onto the image sensor.

## Claims

### Claim 1

An image capture apparatus comprising:
an image sensor which has a tunable spectral response and which is tunable in accordance with a capture mask to capture multi-spectral image data; and
light-field optics arranged to project light-field information of a scene onto the image sensor.

### Claim 2 (depends on 1)

The image capture apparatus according to claim 1, further comprising a mask memory which stores the capture mask for application to the image sensor, so as to tune the spectral response of the image sensor.

### Claim 3 (depends on 2)

The image capture apparatus according to claim 2, wherein the capture mask is configured to tune the image sensor to capture more than three (3) spectral components.

### Claim 4 (depends on 1)

The image capture apparatus according to claim 1, wherein the light-field optics comprise multi-aperture optics.

### Claim 5 (depends on 1)

The image capture apparatus according to claim 1, further comprising a user interface which accepts a user designation of a number of spectral components, wherein the capture mask is constructed in accordance with the user designation and is applied to the image sensor so as to tune the spectral response of the image sensor to the number of spectral components designated by the user.

### Claim 6 (depends on 1)

The image capture apparatus according to claim 1, wherein the image capture apparatus is operable to capture a preview of the scene, and further comprising:
a preview analysis unit constructed to analyze the preview image, to construct the capture mask based on the analysis of the preview image, and to determine an aperture for the light-field optics based on the analysis of the preview image;
wherein the capture mask constructed by the preview analysis unit is applied to the image sensor and the light-field optics are set to the aperture determined by the preview analysis unit, prior to capture of an image of the scene.

### Claim 7 (depends on 1)

The image capture apparatus according to claim 1, further comprising an image memory for storing multi-spectral light-field information captured by the image sensor.

### Claim 8 (depends on 7)

The image capture apparatus according to claim 7, further comprising an interface for transfer of the multi-spectral light-field information to an external image synthesis host, wherein the image synthesis host processes the multi-spectral light-field information into a synthetic image by applying one or more of a desired focal point, a desired color rendering, a desired focal depth, and a desired perspective.

### Claim 9 (depends on 8)

The image capture apparatus according to claim 8, wherein the synthetic image is comprised of tri-stimulus color values with three (3) spectral components.

### Claim 10

A method of image capture for an image capture apparatus, comprising:
capturing multi-spectral image data with an image sensor of the image capture apparatus, wherein the image sensor has a tunable spectral response and is tunable in accordance with a capture mask; and
projecting light-field information of a scene onto the image sensor with light-field optics of the image capture apparatus.

### Claim 11 (depends on 10)

The method according to claim 10, further comprising storing the capture mask in a mask memory for application to the image sensor, so as to tune the spectral response of the image sensor.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the capture mask is configured to tune the image sensor to capture more than three (3) spectral components.

### Claim 13 (depends on 10)

The method according to claim 10, wherein the light-field optics comprise multi-aperture optics.

### Claim 14 (depends on 10)

The method according to claim 10, further comprising providing a user interface which accepts a user designation of a number of spectral components, wherein the capture mask is constructed in accordance with the user designation and is applied to the image sensor so as to tune the spectral response of the image sensor to the number of spectral components designated by the user.

### Claim 15 (depends on 10)

The method according to claim 10, further comprising:
capturing a preview of the scene;
analyzing the preview image;
constructing a capture mask based on the analysis of the preview image; and
determining an aperture for the light-field optics based on the analysis of the preview image,
wherein the constructed capture mask is applied to the image sensor and the light-field optics are set to the aperture determined in the preview analysis, prior to capture of an image of the scene.

### Claim 16 (depends on 10)

The method according to claim 10, wherein the image capture apparatus comprises an image memory for storing multi-spectral light-field information captured by the image sensor.

### Claim 17 (depends on 16)

The method according to claim 16, wherein the image capture apparatus further comprises an interface for transfer of the multi-spectral light-field information to an external image synthesis host, and wherein the image synthesis host processes the multi-spectral light-field information into a synthetic image by applying one or more of a desired focal point, a desired color rendering, a desired focal depth, and a desired perspective.

### Claim 18 (depends on 17)

The method according to claim 17, wherein the synthetic image is comprised of tri-stimulus color values with three (3) spectral components.

### Claim 19

An image processing module for an image capture apparatus, comprising:
a capture module for capturing multi-spectral image data with an image sensor of the image capture apparatus, wherein the image sensor has a tunable spectral response and which is tunable in accordance with a capture mask; and
a projection module for projecting light-field information of a scene with light-field optics of the image capture apparatus, wherein the light-field information is projected onto the image sensor.

### Claim 20 (depends on 19)

The image processing module according to claim 19, wherein the capture mask is stored in a mask memory for application to the image sensor so as to tune the spectral response of the image sensor.

### Claim 21 (depends on 20)

The image processing module according to claim 20, wherein the capture mask is configured to tune the image sensor to capture more than three (3) spectral components.

### Claim 22 (depends on 19)

The image processing module according to claim 19, wherein the light-field optics comprise multi-aperture optics.

### Claim 23 (depends on 19)

The image processing module according to claim 19, further comprising a user interface module for accepting a user designation of a number of spectral components, wherein the capture mask is constructed in accordance with the user designation and is applied to the image sensor so as to tune the spectral response of the image sensor to the number of spectral components designated by the user.

### Claim 24 (depends on 19)

The image processing module according to claim 19, further comprising:
a preview capture module for capturing a preview of the scene;
a preview analysis module constructed to analyze the preview image, to construct the capture mask based on the analysis of the preview image, and to determine an aperture for the light-field optics based on the analysis of the preview image;
wherein the capture mask constructed by the preview analysis module is applied to the image sensor and the light-field optics are set to the aperture determined by the preview analysis module, prior to capture of an image of the scene.

### Claim 25 (depends on 19)

The image processing module according to claim 19, wherein the image capture apparatus comprises an image memory for storing multi-spectral light-field information captured by the image sensor.

### Claim 26 (depends on 25)

The image processing module according to claim 25, wherein the image capture apparatus further comprises an interface for transfer of the multi-spectral light-field information to an external image synthesis host, and wherein the image synthesis host processes the multi-spectral light-field information into a synthetic image by applying one or more of a desired focal point, a desired color rendering, a desired focal depth, and a desired perspective.

### Claim 27 (depends on 26)

The image processing module according to claim 26, wherein the synthetic image is comprised of tri-stimulus color values with three (3) spectral components.

### Claim 28

A computer-readable storage medium storing computer-executable process steps for performing a method image capture for an image capture apparatus, the method comprising:
capturing multi-spectral image data with an image sensor of the image capture apparatus, wherein the image sensor has a tunable spectral response and which is tunable in accordance with a capture mask; and
projecting light-field information of a scene with light-field optics of the image capture apparatus, wherein the light-field information is projected onto the image sensor.

### Claim 29 (depends on 28)

The computer-readable storage medium according to claim 28, wherein the method further comprises storing the capture mask in a mask memory for application to the image sensor, so as to tune the spectral response of the image sensor.

### Claim 30 (depends on 29)

The computer-readable storage medium according to claim 29, wherein the capture mask is configured to tune the image sensor to capture more than three (3) spectral components.

### Claim 31 (depends on 28)

The computer-readable storage medium according to claim 28, wherein the light-field optics comprise multi-aperture optics.

### Claim 32 (depends on 28)

The computer-readable storage medium according to claim 28, wherein the method further comprises providing a user interface which accepts a user designation of a number of spectral components, wherein the capture mask is constructed in accordance with the user designation and is applied to the image sensor so as to tune the spectral response of the image sensor to the number of spectral components designated by the user.

### Claim 33 (depends on 28)

The computer-readable storage medium according to claim 28, wherein the method further comprises:
capturing a preview of the scene;
analyzing the preview image;
constructing a capture mask based on the analysis of the preview image; and
determining an aperture for the light-field optics based on the analysis of the preview image,
wherein the constructed capture mask is applied to the image sensor and the light-field optics are set to the aperture determined in the preview analysis, prior to capture of an image of the scene.

### Claim 34 (depends on 28)

The computer-readable storage medium according to claim 28, wherein the image capture apparatus comprises an image memory for storing multi-spectral light-field information captured by the image sensor.

### Claim 35 (depends on 34)

The computer-readable storage medium according to claim 34, wherein the image capture apparatus further comprises an interface for transfer of the multi-spectral light-field information to an external image synthesis host, and wherein the image synthesis host processes the multi-spectral light-field information into a synthetic image by applying one or more of a desired focal point, a desired color rendering, a desired focal depth, and a desired perspective.

### Claim 36 (depends on 35)

The computer-readable storage medium according to claim 35, wherein the synthetic image is comprised of tri-stimulus color values with three (3) spectral components.
