Passive and single-viewpoint 3D imaging system
AssigneesWilliam Marsh Rice University, Carnegie Mellon University
InventorsYicheng Wu, Vivek Boominathan, Huaijin Chen, Aswin C. Sankaranarayanan, Ashok Veeraraghavan
Life of this patent
Estimating the term…
Abstract
A method for a passive single-viewpoint 3D imaging system comprises capturing an image from a camera having one or more phase masks. The method further includes using a reconstruction algorithm, for estimation of a 3D or depth image.
Claims18
A method for a passive single-viewpoint 3D imaging system, comprising:
- capturing an image from a camera, the camera having one or more phase masks;
- using a reconstruction algorithm, for estimation of a 3D or depth image,
- wherein the reconstruction algorithm comprises:
- building a differentiable forward model with an optical layer and a reconstruction network,
- wherein in the optical layer, a physics-based model simulates a coded image captured on a sensor with a phase mask, and
- wherein in the reconstruction network, a neural network produces a depth map from the coded image.
A system for a passive and single-viewpoint 3D imaging system, comprising:
- a camera;
- and
- an optimized phase mask placed in an aperture plane of the camera,
- wherein the optimized phase mask and a reconstruction algorithm allow the system to estimate a 3D or depth image,
- wherein the reconstruction algorithm comprises a computer processor configured to:
- build a differentiable forward model with an optical layer comprising a physics-based model to simulate coded image captured on a sensor with the phase mask and a neural network to produce a depth map from the coded image;
- train a framework with a synthetic red green blue depth (RGBD) dataset to optimize a height map of the phase mask and parameters in the neural network;
- calibrate a plurality of point spread functions (PSFs) at different depths;
- simulate the coded image with the calibrated PSFs to fine-tune the neural network;
- and
- recover a corresponding depth map by capturing and passing an image through the neural network.
A non-transitory computer readable medium storing instructions, the instructions executable by a computer processor and comprising functionality for:
- building a differentiable forward model with an optical layer and a reconstruction network,
- wherein in the optical layer, a physics-based model simulates a coded image captured on a sensor with a phase mask, and
- wherein in the reconstruction network, a neural network produces a depth map from the coded image;
- training a framework with a red green blue depth (RGBD) dataset to optimize a height map of the phase mask and parameters in the neural network;
- fabricating an optimal phase mask using photolithography method and placing in an aperture plane of a camera;
- calibrating a plurality of point spread functions (PSFs) at different depths;
- simulating the coded image with the calibrated PSFs to fine-tune the neural network;
- and
- recovering a corresponding depth map by capturing and passing an image through the neural network.
Dependencies are inferred from the claim text.