Spatial three-dimensional video generation from two-dimensional content
AssigneeMeta Platforms, Inc.
InventorsNaina Dhingra, Bo Zhu
Life of this patent
Estimating the term…
Abstract
Methods and system to generate a 3D representation of a video frame from a single 2D input frame are provided. The system may obtain a 2D input frame including objects. The system may apply a machine learning model including a depth map to utilize the depth map to generate depth content of the objects of the 2D input frame. The system may generate, based on the depth content of the objects and the 2D input frame, a first output frame of the objects. The system may generate, based on the depth content of the objects and the 2D input frame, a second output frame of the objects. The system may provide, by utilizing the first output frame and the second output frame, a 3D representation of the objects of the 2D input frame.
Claims20
A method comprising:
- obtaining a two-dimensional (2D) input frame comprising objects;
- applying a machine learning model comprising a depth map to utilize the depth map to generate depth content of the objects of the 2D input frame;
- generating, based on the depth content of the objects and the 2D input frame, a first output frame of the objects;
- generating, based on the depth content of the objects and the 2D input frame, a second output frame of the objects;
- and
- providing, by utilizing the first output frame and the second output frame, a three-dimensional (3D) representation of the objects of the 2D input frame.
An apparatus comprising:
- one or more processors;
- and
- at least one memory storing instructions, that when executed by the one or more processors, cause the apparatus to:
- obtain a two-dimensional (2D) input frame comprising objects;
- apply a machine learning model comprising a depth map to utilize the depth map to generate depth content of the objects of the 2D input frame;
- generate, based on the depth content of the objects and the 2D input frame, a first output frame of the objects;
- generate, based on the depth content of the objects and the 2D input frame, a second output frame of the objects;
- and
- provide, by utilizing the first output frame and the second output frame, a three-dimensional (3D) representation of the objects of the 2D input frame.
A non-transitory computer-readable medium storing instructions that, when executed, cause:
- obtaining a two-dimensional (2D) input frame comprising objects;
- applying a machine learning model comprising a depth map to utilize the depth map to generate depth content of the objects of the 2D input frame;
- generating, based on the depth content of the objects and the 2D input frame, a first output frame of the objects;
- generating, based on the depth content of the objects and the 2D input frame, a second output frame of the objects;
- and
- providing, by utilizing the first output frame and the second output frame, a three-dimensional (3D) representation of the objects of the 2D input frame.
Dependencies are inferred from the claim text.