Systems and methods for determining optical flow
AssigneeFacebook, Inc.
InventorBalamanohar Paluri
Life of this patent
Estimating the term…
Abstract
Systems, methods, and non-transitory computer-readable media can obtain a first video frame and a second video frame. The first video frame can be processed using a convolutional neural network to output a first set of feature maps. The second video frame can be processed using the convolutional neural network to output a second set of feature maps. The first set of feature maps and the second set of feature maps can be processed using a spatial matching layer of the convolutional neural network to determine an optical flow for at least one pixel.
Claims21
1-20. (canceled)
A computer-implemented method comprising:
- obtaining, by a computing system, a first video frame and a second video frame;
- processing, by the computing system, the first video frame using a convolutional neural network to output a first set of feature maps, wherein the convolutional neural network includes at least a first convolutional layer and a second convolutional layer, wherein signals outputted by the first convolutional layer are processed by a first pooling layer and then inputted to the second convolutional layer;
- processing, by the computing system, the second video frame using the convolutional neural network to output a second set of feature maps;
- processing, by the computing system, the first set of feature maps and the second set of feature maps by the convolutional neural network to determine an optical flow for at least one pixel.
A system comprising:
- at least one processor;
- and
- a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
- obtaining a first video frame and a second video frame;
- processing the first video frame using a convolutional neural network to output a first set of feature maps, wherein the convolutional neural network includes at least a first convolutional layer and a second convolutional layer, wherein signals outputted by the first convolutional layer are processed by a first pooling layer and then inputted to the second convolutional layer;
- processing the second video frame using the convolutional neural network to output a second set of feature maps;
- processing the first set of feature maps and the second set of feature maps by the convolutional neural network to determine an optical flow for at least one pixel.
A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
- obtaining a first video frame and a second video frame;
- processing the first video frame using a convolutional neural network to output a first set of feature maps, wherein the convolutional neural network includes at least a first convolutional layer and a second convolutional layer, wherein signals outputted by the first convolutional layer are processed by a first pooling layer and then inputted to the second convolutional layer;
- processing the second video frame using the convolutional neural network to output a second set of feature maps;
- processing the first set of feature maps and the second set of feature maps by the convolutional neural network to determine an optical flow for at least one pixel.
Dependencies are inferred from the claim text.