US 12,705,773 B2 · System and method of unsupervised stereo matching with surface normal assistance for indoor applications

Abstract

A system and method for unsupervised stereo matching with surface normal assistance for indoor applications. According to the disclosure, a deep neural network with a feature extraction module, a normal branch, and a disparity branch is disclosed. The extraction module and the normal branch are trained first in a supervised manner for surface normal prediction. The predicted surface normal is then incorporated into the disparity branch, which is trained later in an unsupervised manner for disparity estimation. The latter unsupervised learning approach can reduce our method's dependence on a large amount of ground truth data that is difficult to collect. Experimental results indicate that our proposed method can predict accurate surface normal at textureless regions. With the help of the surface normal, the predicted disparity at these challenging areas is more accurate, which leads to improved quality of stereo matching in indoor scenarios.

Claims6

Dependent claims

A system for unsupervised stereo matching of surfaces for a semi-autonomous cleaning apparatus comprising:

a processor;
one or more cameras configured to capture stereo images;
a neural network software module, the neural network software module further comprising:
a feature extraction module;
a normal branch module;
and a disparity branch module;
wherein the extraction module and the normal branch module are trained in a supervised manner for surface normal prediction;
wherein the surface normal prediction is then incorporated into the disparity branch module which is trained in an unsupervised manner for predicted disparity estimation;
wherein the predicted surface normal is configured to estimate disparity at textureless regions;
wherein the predicted disparity estimation is configured for accurate and improved quality of stereo matching of indoor surfaces by estimating the surface normal accurately at the textureless regions, leading to improvement in the disparity-estimation accuracy and stereo-matching quality in indoor applications involving the textureless regions.

7 more elements

Dependencies are inferred from the claim text.

System and method of unsupervised stereo matching with surface normal assistance for indoor applications

Life of this patent

System and method of unsupervised stereo matching with surface normal assistance for indoor applications

Life of this patent

Abstract

Claims6