Method for estimating high-quality depth maps based on depth prediction and enhancement subnetworks

A method for estimating high-quality depth map based on depth prediction and enhancement sub-networks, wherein comprising the following steps:

• (1) preparing initial data, including color images and corresponding depth maps for training and testing;
• (2) constructing the depth prediction sub-network:
• replacing the last two down-sampling layers of ResNet-101 (residue network with 101 layers) with dilated convolution with dilation rate 2-, obtaining the preliminary depth prediction sub-network;
• and
• replacing the final full connected layer of the preliminary depth prediction sub-network with four dilated convolutions of different dilation rates to formulate a spatial pyramid pooling scheme;
• the dilation rates are 6,12,18,24, separately, obtaining the depth prediction sub-network;
• (3) constructing the depth enhancement sub-network:
• using high-frequency filter to extract the high-frequency information of the color image and using several convolutional layers to extract the features of the high-frequency information as one of the inputs of depth enhancement sub-network;
• the input branch of depth map is a CNN structure comprising three same up-sampling modules, each including a convolutional layer, a standard skip structure and a deconvolutional layer;
• each module up-samples the resolution of depth map at a scale of 2;
• and
• concatenating the above two branches together and using a convolution operation to obtain the final output depth map;
• obtaining the depth enhancement sub-network;
• and
• (4) training the whole network composed by step (2) and step (3).