View synthesis
AssigneeNVIDIA Corporation
InventorsDeqing Sun, Orazio Gallo, Jan Kautz, Jinwei Gu, Wei-Sheng Lai
Life of this patent
Estimating the term…
Abstract
A video stitching system combines video from different cameras to form a panoramic video that, in various embodiments, is temporally stable and tolerant to strong parallax. In an embodiment, the system provides a smooth spatial interpolation that can be used to connect the input video images. In an embodiment, the system applies an interpolation layer to slices of the overlapping video sources, and the network learns a dense flow field to smoothly align the input videos with spatial interpolation. Various embodiments are applicable to areas such as virtual reality, immersive telepresence, autonomous driving, and video surveillance.
Claims24
A method, comprising:
- generating a first frame from a first video and a second frame from a second video captured by cameras at different locations, the respective frames having overlapping fields of view;
- slicing the overlapping fields of view of corresponding frames into a plurality of corresponding image slices of respective overlap regions of the corresponding frames;
- determining a forward flow field from the first overlap region to the second overlap region and a reverse flow field from the second overlap region to the first overlap region and scale the forward and reverse flow fields of respective slices to generate an interpolated region;
- and
- combining the interpolated overlap region with non-overlapping regions of the respective frames slices to form a panoramic frame.
An apparatus, comprising:
- one or more processors to:
- generate a first frame from a first video and a second frame from a second video captured by cameras at different locations, the respective frames having overlapping fields of view;
- slice the overlapping fields of view of corresponding frames into a plurality of corresponding image slices of respective overlap regions of the corresponding frames;
- determine a forward flow field from the first overlap region to the second overlap region and a reverse flow field from the second overlap region to the first overlap region and scale the forward and reverse flow fields of respective slices to generate an interpolated region;
- and
- combine the interpolated region with non-overlapping regions of the respective frames slices to form a panoramic frame.
One or more processors, comprising:
- circuitry to:
- generate a first frame from a first video and a second frame from a second video captured by cameras at different locations, the respective frames having overlapping fields of view;
- slice the overlapping fields of view of corresponding frames into a plurality of corresponding image slices of respective overlap regions of the corresponding frames;
- determine a forward flow field from the first overlap region to the second overlap region and a reverse flow field from the second overlap region to the first overlap region and scale the forward and reverse flow fields of respective slices to generate an interpolated region;
- and
- combine the interpolated region with non-overlapping regions of the respective frames slices to form a panoramic frame.
Dependencies are inferred from the claim text.