Capture of raw augmented reality data and subsequent 3d mesh generation
AssigneeStreem, LLC
InventorsFlora Ponjou Tasse, Pavan Kumar Kamaraju, Ghislain Fouodji Tasse, Ryan R. Fink, Sean M. Adkinson
Life of this patent
Estimating the term…
Abstract
Embodiments include systems and methods for generating a 3D mesh from a video stream or other image captured contemporaneously with AR data. The AR data is used to create a depth map, which is then fused with images from frames of the video to form a full 3D mesh. The images and depth map can also be used with an object detection algorithm to recognize 3D objects within the 3D mesh. Methods for fingerprinting the video with AR data captured contemporaneously with each frame are disclosed.
Claims15
A method for generation of a 3D mesh from a video stream, comprising:
- capturing a video stream and AR data synchronized to the video stream, wherein each video stream is comprised a plurality of frames, each frame including image data;
- generating, with the video stream and AR data, a depth map for each frame of the plurality of frames of the video stream;
- incrementally generating, from the depth map and image data for each frame of the plurality of frames, a 3D volume representing an environment captured by the video stream;
- and
- generating, from the 3D volume, the 3D mesh.
A non-transitory computer readable medium (CRM) comprising instructions that, when executed by an apparatus, cause the apparatus to:
- capture a video stream comprised of a plurality of frames, each frame including image data;
- capture AR data corresponding to each frame of the plurality of frames;
- generate, from the AR data, a depth map for each frame of the plurality of frames;
- construct, from the depth map of each frame, a 3D volume;
- and
- generate, from the 3D volume, a 3D mesh.
Dependencies are inferred from the claim text.