Lidar stereo fusion live action 3D model video reconstruction for six degrees of freedom 360° volumetric virtual reality video
AssigneeHypeVR
InventorsAnthony Tran, Jason Juang
Life of this patent
Estimating the term…
Abstract
A system for capturing live-action three-dimensional video is disclosed. The system includes pairs of stereo cameras and a LIDAR for generating stereo images and three-dimensional LIDAR data from which three-dimensional data may be derived. A depth-from-stereo algorithm may be used to generate the three-dimensional camera data for the three-dimensional space from the stereo images and may be combined with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.
Claims24
A system for capturing live action three-dimensional video comprising:
- a first and a second camera, making up a stereo camera pair, the first and second stereo cameras each having known imaging parameters such that a depth-from-stereo algorithm may be applied to corresponding sets of images captured using the stereo camera pair to generate three-dimensional camera data for a three-dimensional space viewed by the stereo camera pair;
- a LIDAR, having a known position relative to the stereo camera pair, configured to produce three-dimensional LIDAR data;
- a time-stamp device for applying a first time-stamp to the three-dimensional camera data and a second time-stamp to the three-dimensional LIDAR data;
- and
- a controller configured to
- use the depth-from-stereo algorithm to generate the three-dimensional camera data for the three-dimensional space, and
- combine the three-dimensional camera data and the three-dimensional LIDAR data, with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.
A system for capturing live action three-dimensional video comprising:
- a plurality of stereo camera pairs, each made up of a first stereo camera and a second stereo camera each camera having known imaging parameters such that a depth-from-stereo algorithm may be applied to corresponding sets of images captured using each of the plurality of stereo camera pairs to generate three-dimensional camera data for a three-dimensional space viewed by the plurality of stereo camera pairs;
- a LIDAR, having a known position relative to the stereo camera pair, configured to produce three-dimensional LIDAR data;
- a time-stamp device for applying a first time-stamp to the three-dimensional camera data and a second time-stamp to the three-dimensional LIDAR data;
- and
- a controller configured to
- use the depth-from-stereo algorithm to generate the three-dimensional camera data for the three-dimensional space, and
- combine the three-dimensional camera data and the three-dimensional LIDAR data, with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.
Apparatus comprising a non-transitory storage medium storing a program having instructions which when executed by a processor will cause the processor to generate three-dimensional data, the instructions of the program for:
- generating three-dimensional camera data for a three-dimensional space viewed by a plurality of stereo camera pairs, each made up of a first stereo camera and a second stereo camera each camera having known imaging parameters such that a depth-from-stereo algorithm may be applied to corresponding sets of images;
- producing three-dimensional LIDAR data using a LIDAR;
- applying a first time-stamp to the three-dimensional camera data and a second time-stamp to the three-dimensional LIDAR data;
- using the depth-from-stereo algorithm to generate the three-dimensional camera data for the three-dimensional space;
- and
- combining the three-dimensional camera data and the three-dimensional LIDAR data, with the three-dimensional LIDAR data taking precedence over the three-dimensional camera data to thereby generate three-dimensional data corresponding to the three-dimensional space.
Dependencies are inferred from the claim text.