Real-time stereo matching using a hierarchical iterative refinement network

A method for real-time stereo matching comprising:

• extracting, by a feature extractor, features from first and second images at a plurality of resolutions including a first resolution and a second resolution;
• generating, by an initialization engine, a plurality of depth estimations at the plurality of resolutions, including:
• predicting a first depth estimation based on a matching of the features from the first and second images at the first resolution;
• and
• predicting a second depth estimation based on a matching of the features from the first and second images at the second resolution;
• and
• iteratively refining, by a propagation engine, the plurality of depth estimations based on image warping and spatial propagation, including:
• predicting, via a first iteration, a refined first depth estimation using the first depth estimation and the extracted features at the first resolution;
• and
• predicting, via a second iteration, a refined second depth estimation based on the refined first depth estimation from the first iteration and the second depth estimation, the refined second depth estimation being used in a subsequent iteration or as a basis for a depth map.

A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to:

• receive a first image captured from a first camera and a second image captured from a second camera;
• extract features from the first and second images at a plurality of resolutions including a first resolution and a second resolution;
• generate a plurality of depth estimations at the plurality of resolutions using the extracted features, the plurality of depth estimations including a first depth estimation and a second depth estimation;
• and
• iteratively refine the depth estimations based on image warping and spatial propagation, including:
• predict, via a first iteration, a refined first depth estimation using the first depth estimation and the extracted features at the first resolution;
• and
• predict, via a second iteration, a refined second depth estimation based on the refined first depth estimation from the first iteration, and the second depth estimation, the refined second depth estimation being used in a subsequent iteration or as a basis for a depth map.