Real-time Photorealistic 3D Holography With Deep Neural Networks
AssigneeMassachusetts Institute of Technology
InventorsWojciech Matusik, Liang Shi
Life of this patent
Estimating the term…
Abstract
A method for producing a hologram representative of a subject three-dimensional scene includes receiving and storing input digital data characterizing a first image of the subject three-dimensional scene. The method further includes processing the data in a neural network that has been trained to transform the input digital data into a holographic representation of the subject three-dimensional scene, the representation containing phase information characterizing depth and parallax of the scene. The method also includes providing an output of the holographic representation of the subject three-dimensional scene.
Claims26
A method for producing a hologram representative of a subject three-dimensional scene, the method comprising:
- receiving and storing input digital data characterizing a first image of the subject three-dimensional scene;
- processing the data in a neural network that has been trained to transform the input digital data into a holographic representation of the subject three-dimensional scene, the representation containing phase information characterizing depth and parallax of the scene;
- and
- providing an output of the holographic representation of the subject three-dimensional scene.
(canceled)
(canceled)
(canceled)
(canceled)
A method for producing a hologram representative of a subject three-dimensional scene, the method comprising:
- receiving and storing input digital data characterizing a set of images of the subject three-dimensional scene;
- processing the data in a neural network that has been trained to transform the input digital data into a phase-only holographic representation of the subject three-dimensional scene, the representation containing phase information characterizing depth and parallax of the scene, wherein the representation is computed to take into account effects of wave-based occlusion with respect to the complete scene;
- and
- providing an output of the holographic representation of the subject three-dimensional scene.
(canceled)
A method for producing a hologram representative of a subject three-dimensional scene, the method comprising:
- receiving and storing input digital data characterizing a set of images of the subject three-dimensional scene;
- processing the data in a neural network that has been trained to transform the input digital data into a phase-only holographic representation of the subject three-dimensional scene and has been further trained in two stages to directly optimize the phase-only hologram (with anti-aliasing processing) by incorporating a complex to phase-only conversion into the training,
- wherein in a first stage the neural network is trained to predict a midpoint hologram propagated to a center of the subject three-dimensional scene and to minimize a difference between a target focal stack and a predicted focal stack, and
- in a second stage a phase-only target hologram is generated from the predicted midpoint hologram and refined by calculating a dynamic focal stack loss, between a post-encoding focal stack and the target focal stack, and a regularization loss associated therewith, the representation containing phase information characterizing depth and parallax of the scene;
- and
- providing an output of the holographic representation of the subject three-dimensional scene.
(canceled)
Dependencies are inferred from the claim text.