Spatio-temporal segmentation for video
AssigneeGoogle Inc.
InventorsMatthias Grundmann, Vivek Kwatra, Mei Han
Life of this patent
Estimating the term…
Abstract
A video is segmented to produce volumetric video regions. Descriptors are created for the video regions. A region graph is created for the video, where the region graph has weighted edges incident to video regions and the weight of an edge is calculated responsive to the descriptors of the video regions incident to the edge. The region graph is segmented responsive to the weights of the edges incident to the video regions to produce a new region graph having new volumetric video regions comprised of merged video regions of the first region graph. The descriptions of the region graphs are stored in a data storage.
Claims24
A method of using a computer to segment a video, comprising:
- segmenting the video to produce volumetric video regions;
- creating descriptors for the video regions;
- generating a first region graph for the video, the first region graph having weighted edges incident to video regions, the weight of an edge calculated responsive to the descriptors of the video regions incident to the edge;
- determining an internal variation of a first video region;
- determining an internal variation of a second video region;
- determining a weight of an edge incident to the first and second video regions;
- merging the first and second video regions responsive to a determination that the weight of the edge incident to the first and second video regions is less than the internal variations of the first and second video regions to produce a second region graph comprising at least the merged first and second video region;
- and
- storing descriptions of the first and second region graphs in a data storage.
A computer for segmenting a video, the computer comprising:
- a computer-readable storage medium storing executable computer program instructions comprising instructions for:
- segmenting the video to produce volumetric video regions;
- creating descriptors for the video regions;
- generating a first region graph for the video, the first region graph having weighted edges incident to video regions, the weight of an edge calculated responsive to the descriptors of the video regions incident to the edge;
- determining an internal variation of a first video region;
- determining an internal variation of a second video region;
- determining a weight of an edge incident to the first and second video regions;
- merging the first and second video regions responsive to a determination that the weight of the edge incident to the first and second video regions is less than the internal variations of the first and second video regions to produce a second region graph comprising at least the merged first and second video region;
- and
- storing descriptions of the first and second region graphs in a data storage;
- and
- a processor configured to execute the computer program instructions stored on the computer-readable storage medium.
A non-transitory computer-readable storage medium storing executable computer program instructions for segmenting a video, the instructions comprising instructions for:
- segmenting the video to produce volumetric video regions;
- creating descriptors for the video regions;
- generating a first region graph for the video, the first region graph having weighted edges incident to video regions, the weight of an edge calculated responsive to the descriptors of the video regions incident to the edge;
- determining an internal variation of a first video region;
- determining an internal variation of a second video region;
- determining a weight of an edge incident to the first and second video regions;
- merging the first and second video regions responsive to a determination that the weight of the edge incident to the first and second video regions is less than the internal variations of the first and second video regions to produce a second region graph comprising at least the merged first and second video region;
- and
- storing descriptions of the first and second region graphs in a data storage.
Dependencies are inferred from the claim text.