Disparity estimation optimization method based on upsampling and exact rematching

A disparity estimation optimization method based on upsampling and exact rematching, comprising the following steps:

• step 1:
• extracting discriminable features;
• step 2:
• conducting initial cost matching and cost map optimization to obtain an initial disparity map with low resolution;
• step 3:
• obtaining a disparity map with one resolution higher from the initial disparity map with low resolution by a propagation upsampling method and an exact rematching method, and repeating the process until the original resolution is restored;
• 3.1 the propagation upsampling method
• the initial disparity map Dn+1 with minimum resolution is first subjected to interpolation and upsampling to obtain a coarsely matched disparity map D′n, the disparity map obtained at this moment is only obtained by numerical interpolation without reference to any structural information of an original image, a left view is reestablished with an original right view Ir according to the coarsely matched disparity map D′n and denoted as Ĩl, and then the error between the reestablished left view Ĩl and a real left view Il is calculated to obtain a confidence map Mc:
• Mc=1−normalization(Il−Ĩl) (2)
• normalization(⋅) is normalized operation, the difference is normalized to (0,1), and the probability value at each point on the confidence map Mc represents the confidence of the disparity value of the pixel;
• and the confidence map is reproduced and translated to become a confidence map group which is denoted as Mcg,
• Mcg=fc(Mc,k,s) (3)
• wherein fc(⋅) represents the operation of reproduction and translation to resize, k represents the size of a neighboring window, and s represents the void content of a sampling window;
• and the receptive field is (2s+1)2, and a confidence vector of k*k is obtained at each position, which represents the confidence of a pixel in a k*k neighboring window around the pixel;
• a relative relation network module is proposed, a left feature map with the corresponding resolution is input into the module, and a weight vector is worked out at each position, which indicates the relative relation of the neighboring pixel and the center pixel, i.e., the larger the weight is, the greater the effect of a neighboring pixel on the pixel is;
• and the weight is donated as Wrelative;
• Wrelative=relative(Fnl,k) (4)
• wherein k represents the size of a neighboring window, and relative represents the relative relation network module;
• the coarsely matched disparity map D′n, the confidence map Mcg and the relative relation weight Wrelative are used for propagation to obtain a propagated disparity map, and the propagation calculation process is as follows:
• Dnp=<fc(D′n,k,s),softmax(Wrelative*Mcg)> (5)
• wherein Dnp represents the propagated disparity map, <, > represents dot product operation, fc(⋅) represents the operation of reproduction and translation to resize, and softmax(Wrelative*Mcg) represents the support strength of the surrounding pixel to the center pixel during propagation and is obtained by multiplying the confidence of the surrounding pixel and the relative relation weight;
• then the void content of the window is used for repeating the propagation process so that the optimized disparity map can be propagated in different receptive fields;
• and at this point, the propagation upsampling process from Dn+1 to Dnp is completed;
• 3.2 the exact rematching method
• first, a left feature map is reestablished with a right feature map Fnr with the corresponding resolution in a feature list according to Dnp and donated as , and =fw(Fnr, Dnp);
• and rematching is conducted once with the reestablished left feature map and the original left feature map Fnl within a small range of the disparity d=[−d0, d0] to obtain a cost map, then the cost map is optimized through an hourglass network, the disparity is regressed to obtain a bias map Δ which represents an offset from Dnp, and the two maps are added to obtain a final disparity map Dn of an optimized network;
• Dn=Dnp+Δ (6)
• the processes of 3.1 and 3.2 are iterated repeatedly until the original resolution is restored to obtain a final high-precision disparity map.