Unified simultaneous optical flow and depth estimation

An apparatus for image processing, comprising:

• at least one memory;
• and
• at least one processor coupled to the at least one memory, the at least one processor being configured to:
• correlate a first set of features obtained from a first viewpoint at a first time period with a second set of features obtained from a second viewpoint at the first time period to generate a first disparity cost volume;
• correlate a third set of features obtained from the first viewpoint at a second time period with the first set of features to generate a first optical flow cost volume;
• gate the first disparity cost volume to generate first intermediate disparity information;
• gate the first optical flow cost volume to generate first intermediate optical flow information;
• correlate the first set of features, the second set of features, and the first intermediate optical flow information to generate disparity information for output;
• and
• correlate the third set of features, the first set of features, and the first intermediate disparity information to generate optical flow information for output.

A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:

• correlate a first set of features obtained from a first viewpoint at a first time period with a second set of features obtained from a second viewpoint at the first time period to generate a first disparity cost volume;
• correlate a third set of features obtained from the first viewpoint at a second time period with the first set of features to generate a first optical flow cost volume;
• gate the first disparity cost volume to generate first intermediate disparity information;
• gate the first optical flow cost volume to generate first intermediate optical flow information;
• correlate the first set of features, the second set of features, and the first intermediate optical flow information to generate disparity information for output;
• and
• correlate the third set of features, the first set of features, and the first intermediate disparity information to generate optical flow information for output.