Systems, methods and devices for augmenting video content
AssigneeThe Board of Trustees of the Leland Stanford Junior University
InventorsAshutosh Saxena, Siddharth Batra, Andrew Y. Ng
Life of this patent
Estimating the term…
Abstract
Methods, systems, products and devices are implemented for editing video image frames. According to one such method, image content is embedded into video. A selection input is received for a candidate location in a video frame of the video. The candidate location is traced in subsequent video frames of the video by approximating three-dimensional camera motion between two frames using a model that compensates for camera rotations, camera translations and zooming, and by optimizing the approximation using statistical modeling of three-dimensional camera motion between video frames. Image content is embedded in the candidate location in the subsequent video frames of the video based upon the tracking thereof.
Claims21
A method for generating video with embedded image content, said method comprising:
- receiving a selection input for a candidate location in a video frame of the video;
- tracking the candidate location in subsequent video frames of the video by
- approximating three-dimensional camera motion between two frames using a model that compensates for camera rotations, camera translations and zooming,
- statistically modeling three-dimensional camera motion between the video frames by estimating and using parameters of a transformation matrix that represents a projective transformation of images in the frame caused by movement of the camera, the projective transformation being based upon the composition of a pair of perspective projections of an image in the video frames, and
- optimizing the approximation using the statistical modeling;
- and
- embedding image content in the candidate location in the subsequent video frames of the video based upon the tracking thereof.
An apparatus comprising:
- an electronic circuit configured and arranged to:
- receive a selection input for a candidate location in a first video frame of the video;
- track the candidate location in subsequent video frames of the video by
- approximating three-dimensional camera motion between two frames,
- statistically modeling the three-dimensional camera motion between the video frames by estimating and using parameters of a transformation matrix that represents a projective transformation of images in the first video frame caused by movement of the camera, the projective transformation being based upon the composition of a pair of perspective projections of an image in the video frames, and
- optimizing the approximation using the statistical modeling of three-dimensional camera motion between video frames;
- and
- embed image content in the candidate location in the subsequent video frames of the video.
A computer product comprising:
- non-transitory computer readable medium storing instructions that when executed perform the steps of:
- receiving a selection input for a candidate location in a video frame of a video;
- tracking the candidate location in subsequent video frames of the video by
- approximating three-dimensional camera motion between two frames using a model that compensates for camera rotations, camera translations and zooming,
- statistically modeling three-dimensional camera motion between the video frames by estimating and using parameters of a transformation matrix that represents a projective transformation of images in the frame caused by movement of the camera, the projective transformation being based upon the composition of a pair of perspective projections of an image in the video frames, and
- optimizing the approximation using the statistical modeling of three-dimensional camera motion between video frames;
- and
- embedding image content in the candidate location in the subsequent video frames of the video based upon the tracking thereof.
A method for generating video with embedded image content, the video including a plurality of temporally-arranged video frames captured by a camera, said method comprising:
- receiving a selection input that identifies the position of a candidate location within a first one of the video frames;
- tracking the position of the candidate location in video frames that are temporally subsequent to the first one of the video frames by
- generating approximation data that approximates three-dimensional motion of the camera between two of the video frames by compensating for rotation, translation and zooming of the camera,
- statistically modeling three-dimensional camera motion between the video frames by estimating and using parameters of a transformation matrix that represents a projective transformation of images in the frame caused by movement of the camera, the projective transformation being based upon the composition of a pair of perspective projections of an image in the video frames,
- modifying the approximation data based on the statistic modeling, and
- using the modified approximation data to determine the position of the candidate location in each of the subsequent video frames;
- and
- embedding image content in the determined position of the candidate location in the subsequent video frames.
Dependencies are inferred from the claim text.