Occlusion-Aware Reconstruction of Three-Dimensional Scenes from Light Field Images
AssigneeRicoh Company, Ltd.
InventorsIvana Tosic, Yenting Lin, Kathrin Berkner
Life of this patent
Estimating the term…
Abstract
A three-dimensional scene is modeled as a set of layers representing different depths of the scene, and as masks representing occlusions of layers by other layers. The layers are represented as linear combinations of atoms selected from an overcomplete dictionary. An iterative approach is used to alternately estimate the atom coefficients for layers from a light field image of the scene, assuming values for the masks, and to estimate the masks given the estimated layers. In one approach, the atoms in the dictionary are ridgelets oriented at different angles, where there is a correspondence between depth and angle.
Claims20
A method for reconstructing a layered form of a three-dimensional scene from a light field of the scene, the method comprising a computer system executing the steps of:
- assuming one or more masks, estimating, based on the scene light field, atom coefficients for layer light fields for two or more layers representing different depths of the scene, wherein the layer light fields are represented as linear combinations of atoms selected from an overcomplete dictionary and the linear combinations are weighted by the atom coefficients, and wherein the masks represent the occlusion of backward layers by forward layers;
- assuming the layer light fields, estimating the masks;
- iteratively performing the steps of estimating atom coefficients for layer light fields and estimating the masks;
- and
- estimating layers of a three-dimensional scene from the layer light fields and the masks.
A non-transitory computer readable medium containing instructions for reconstructing a three-dimensional scene from a light field of the scene, the instructions causing a computer system to execute the steps of:
- assuming one or more masks, estimating, based on the scene light field, atom coefficients for layer light fields for two or more layers representing different depths of the scene, wherein the layer light fields are represented as linear combinations of atoms selected from an overcomplete dictionary and the linear combinations are weighted by the atom coefficients, and wherein the masks represent the occlusion of backward layers by forward layers;
- assuming the layer light fields, estimating the masks;
- iteratively performing the steps of estimating atom coefficients for layer light fields and estimating the masks;
- and
- estimating layers of a three-dimensional scene from the layer light fields and the masks.
A system for reconstructing a three-dimensional scene from a light field of the scene, the system comprising:
- a light field imaging device that captures a light field of the scene;
- and
- a computer system with access to the scene light field, the computer system executing the steps of:
- assuming one or more masks, estimating, based on the scene light field, atom coefficients for layer light fields for two or more layers representing different depths of the scene, wherein the layer light fields are represented as linear combinations of atoms selected from an overcomplete dictionary and the linear combinations are weighted by the atom coefficients, and wherein the masks represent the occlusion of backward layers by forward layers;
- assuming the layer light fields, estimating the masks;
- iteratively performing the steps of estimating atom coefficients for layer light fields and estimating the masks;
- and
- estimating layers of a three-dimensional scene from the layer light fields and the masks.
Dependencies are inferred from the claim text.