Optical flow estimation using 4-dimensional cost volume processing

A system for optical flow estimation, the system comprising:

• a feature extraction circuit to extract a first set of feature vectors from a first image, each feature vector of the first set associated with a pixel of the first image;
• and to extract a second set of feature vectors from a second image, each feature vector of the second set associated with a pixel of the second image;
• a cost volume construction circuit to construct a 4-dimensional (4D) cost volume to store a distance metric between each feature vector of the first set of feature vectors and a selected subset of feature vectors of the second set of feature vectors;
• and
• a cost volume processing circuit to perform a flow-semi-global matching (Flow-SGM) on the 4D cost volume to estimate an optical flow vector for pixels of the first image and to generate an estimated optical flow field comprising the estimated optical flow vectors.

At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for optical flow estimation, the operations comprising:

• extracting a first set of feature vectors from a first image, each feature vector of the first set associated with a pixel of the first image;
• extracting a second set of feature vectors from a second image, each feature vector of the second set associated with a pixel of the second image;
• constructing a 4-dimensional (4D) cost volume to store a distance metric between each feature vector of the first set of feature vectors and a selected subset of feature vectors of the second set of feature vectors;
• and
• performing a flow-semi-global matching (Flow-SGM) on the 4D cost volume to estimate an optical flow vector for pixels of the first image or the second image or both of the first image and second image.