# US 10,484,668 B2 · Imaging apparatus, image processing apparatus, imaging system, imaging method, image processing method, and recording medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,484,668 B2
- Document ID: US10484668B2
- Status: Granted
- Kind: B2
- Application number: 15/431,770
- Filed: 2017-02-14
- Published: 2017-08-31
- Granted: 2019-11-19
- Earliest claimed: 2016-02-29
- Assignees: Panasonic Intellectual Property Management Co., Ltd.
- Applicants: Panasonic Intellectual Property Management Co., Ltd.
- Inventors: Kunio Nobori; Satoshi Sato; Takeo Azuma; Takamasa Ando; Tsuguhiro Korenaga; Toshiya Fujii
- Web page: https://patentgrep.com/patent/US10484668B2

## CPC

- H04N 13/282: for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
- G02B 23/02: involving prisms or mirrors (G02B23/14 takes precedence)
- G02B 27/46: Systems using spatial filters
- G06T 7/11: Region-based segmentation
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- H04N 13/106: Processing image signals (for multi-view video sequence encoding H04N19/597)
- H04N 13/189: Recording image signals; Reproducing recorded image signals
- H04N 13/207: using a single two-dimensional [2D] image sensor
- H04N 13/218: using spatial multiplexing
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An imaging apparatus includes an imaging device, a first imaging optical system and a second imaging optical system that form respective input images from mutually different viewpoints onto the imaging device, and a first modulation mask and a second modulation mask that modulate the input images formed by the first imaging optical system and the second imaging optical system. The imaging device captures a superposed image composed of the two input images that have been formed by the first imaging optical system and the second imaging optical system, modulated by the first modulation mask and the second modulation mask, and optically superposed on each other, and the first modulation mask and the second modulation mask have mutually different optical transmittance distribution characteristics.

## Claims

### Claim 1

An image processing apparatus, comprising:
an acquirer that acquires a superposed image, wherein
two or more imaging optical systems form respective input images from mutually different viewpoints,
two or more modulation masks modulate the input images in modulation modes and output resulting images, and
an imaging device receives the resulting images at a same time and generates the superposed image composed of the input images; and
an image processor that generates reconstructed images corresponding to the input images by splitting the superposed image with the use of modulation information that indicates the modulation modes,
wherein the modulation modes are different,
wherein, when generating the two or more reconstructed images, the image processor
calculates a parallax between the two or more reconstructed images along with the two or more reconstructed images on the basis of an evaluation value, and
calculates a distance to an object in the two or more reconstructed images on the basis of the calculated parallax,
wherein, when generating the two or more reconstructed images, the image processor
calculates a value of a first parameter corresponding to the two or more reconstructed images and a value of a second parameter corresponding to the parallax that minimize the evaluation value, the evaluation value being based on the first parameter and the second parameter,
converts the calculated value of the first parameter to the two or more reconstructed images, and
acquires the calculated value of the second parameter as the parallax,
wherein the evaluation value is a sum of respective values indicated by a first term, a second term, and a third term,
wherein the first term indicates, by using the first parameter, a sum of squares of a difference between the superposed image and images obtained by modulating the two or more reconstructed images in accordance with the modulation information,
wherein the second term indicates a value obtained by weighting an L1-norm of the first parameter, and
wherein the third term indicates, by using the first parameter and the second parameter, a value obtained by weighting an L1-norm of a difference between an image obtained by translating one of the two or more reconstructed images by a distance corresponding to the parallax and another one of the two or more reconstructed images.