US 9,961,321 B2Granted

Image processing device and image processing method having function for reconstructing multi-aspect images, and recording medium

AssigneeCasio Computer Co., Ltd.

InventorKouichi Nakagome

Life of this patent

Estimating the term…

Loading patent…

Abstract

A video is generated from multi-aspect images. For each frame of the output video, a provisional video is created with provisional images which are used to designate a position of a target object on which to focus and to acquire a depth map showing a depth of the object, and which are reconstructed using default values from LFIs as the frames. Then, information designating coordinates of focal positions of the target object on the provisional video is acquired. A list creator acquires the depth value of the target object using a depth map created from the LFI of the current frame, and records this in a designation list after obtaining the reconstruction distance from the depth value. A corrector corrects the designation list. A main video creator creates and outputs a main video with reconstructed images focused at a focal length designated by the post-correction designation list as frames.

Claims23

Dependent claims

An image processing device comprising:

a provisional acquirer that acquires, for each frame of a video, a provisional image for designating a position in the frame of an object on which to focus and for acquiring a depth map showing a depth of the object, each provisional image being reconstructed using a default value for provisional image creation from a multi-aspect image in which the object is shot from each of multiple viewpoints, and the multi-aspect image being a light field image that comprises multiple sub-images;
a coordinate acquirer that acquires a coordinate of a focal position of the object on a provisional image plane, for each provisional image acquired by the provisional acquirer;
a first setter that sets, for a frame of the video, a coordinate of the focal position acquired by the coordinate acquirer in a coordinate system including a component of a normal line axis of the frame;
a second setter that resets the coordinate of the focal position for the frame of the video, based on coordinates of the focal position set by the first setter for other frames of the video;
and
an image acquirer that acquires, for each frame of the video, an image that is (i) reconstructed from the multi-aspect image in which the object is shot from each of the multiple viewpoints, and (ii) focused at a focal length determined from the coordinate of the focal position set by at least one of the first and the second setters.

3 more elements

An imaging device, comprising:

an imager;
a provisional acquirer that acquires, for each frame of a video, a provisional image for designating a position in the frame of an object on which to focus and for acquiring a depth map showing a depth of the object, each provisional image being reconstructed using a default value for provisional image creation from a multi-aspect image in which the object is shot from each of multiple viewpoints, and the multi-aspect image being a light field image that comprises multiple sub-images;
a coordinate acquirer that acquires a coordinate of the focal position of the object on a provisional image plane, for each provisional image acquired by the provisional acquirer;
a first setter that sets, for a frame of the video, a coordinate of the focal position acquired by the coordinate acquirer in a coordinate system including a component of a normal line axis of the frame;
a second setter that resets the coordinate of the focal position for the frame of the video, based on coordinates of the focal position set by the first setter for other frames of the video;
and
a reconstructor that reconstructs, for each frame of the video, an image that is focused at a focal length determined from the coordinate of the focal position set by at least one of the first and second setters, from the multi-aspect image in which the object is shot from each of the multiple viewpoints.

4 more elements

A non-transitory computer-readable recording medium having stored thereon a program executable by a computer, the program controlling the computer to perform functions comprising:

a provisional acquisition function that acquires, for each frame of a video, a provisional image for designating a position in the frame of an object on which to focus and for acquiring a depth map showing a depth of the object, each provisional image being reconstructed using a default value for provisional image creation from a multi-aspect image in which the object is shot from each of multiple viewpoints, and the multi-aspect image being a light field image that comprises multiple sub-images;
a coordinate acquisition function that acquires a coordinate of a focal position of the object on a provisional image plane, for the each provisional image acquired by the provisional acquisition function;
a first setting function that sets, for a frame of the video, a coordinate of the focal position acquired by the coordinate acquisition function in a coordinate system including a component of a normal line axis of the frame;
a second setting function that resets the coordinate of the focal position for the frame of the video based on coordinates of the focal position set by the first setting function for other frames of the video;
and
an image acquisition function that acquires, for each frame of the video, an image that is (i) reconstructed from the multi-aspect image in which the object is shot from each of the multiple viewpoints, and (ii) focused at a focal length determined from the coordinate of the focal position set by at least one of the first and the second setting functions.

3 more elements

Dependencies are inferred from the claim text.