Learnable Cost Volume for Determining Pixel Correspondence

A system comprising:

• a processor;
• and
• a non-transitory computer-readable storage medium having stored thereon instructions that, when executed by the processor, cause the processor to perform operations comprising:
• obtaining (i) a first plurality of feature vectors associated with a first image and (ii) a second plurality of feature vectors associated with a second image;
• generating a plurality of transformed feature vectors by transforming each respective feature vector of the first plurality of feature vectors by a kernel matrix trained to define an elliptical inner product space;
• generating a cost volume by determining, for each respective transformed feature vector of the plurality of transformed feature vectors, a plurality of inner products, wherein each respective inner product of the plurality of inner products is between the respective transformed feature vector and a corresponding candidate feature vector of a corresponding subset of the second plurality of feature vectors;
• and
• determining, based on the cost volume, a pixel correspondence between the first image and the second image.

A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a computing system, cause the computing system to perform operations comprising:

• obtaining (i) a first plurality of feature vectors associated with a first image and (ii) a second plurality of feature vectors associated with a second image;
• generating a plurality of transformed feature vectors by transforming each respective feature vector of the first plurality of feature vectors by a kernel matrix trained to define an elliptical inner product space;
• generating a cost volume by determining, for each respective transformed feature vector of the plurality of transformed feature vectors, a plurality of inner products, wherein each respective inner product of the plurality of inner products is between the respective transformed feature vector and a corresponding candidate feature vector of a corresponding subset of the second plurality of feature vectors;
• and
• determining, based on the cost volume, a pixel correspondence between the first image and the second image.