Transferring salient depth properties from labeled data to unlabeled datasets for monocular depth estimation

A method of performing monocular depth estimation (MDE), the method comprising:

• obtaining an input image;
• and
• generating an estimated depth map by providing the input image to an MDE network,
• wherein a training process for the MDE network comprises:
• obtaining a source dataset comprising a first source image and a first ground truth depth map corresponding to the first source image;
• obtaining a target dataset comprising a first target image and a second target image;
• generating an estimated first source depth map corresponding to the first source image using the MDE network;
• generating an estimated target depth map corresponding to the first target image using the MDE network;
• generating a first estimated relative pose based on the first target image and the second target image using a pose network;
• and
• training the MDE network and the pose network by performing mixed supervision training,
• wherein the performing the mixed supervision training comprises performing fully-supervised training based on the estimated first source depth map and the first ground truth depth map, and performing self-supervised training based on the estimated target depth map and the first estimated relative pose.

A device for training a monocular depth estimation (MDE) network, the device comprising:

• an MDE network configured to generate an estimated depth map based on an input image;
• a pose network configured to estimate a relative pose corresponding to one input image with respect to another input image;
• and
• a training module configured to:
• obtain a source dataset comprising a first source image and a first ground truth depth map corresponding to the first source image,
• obtain a target dataset comprising a first target image and a second target image,
• generate an estimated first source depth map corresponding to the first source image using the MDE network,
• generate an estimated target depth map corresponding to the first target image using the MDE network,
• generate a first estimated relative pose based on the first target image and the second target image using the pose network, and
• train the MDE network and the pose network by performing mixed supervision training including fully-supervised training based on the estimated first source depth map and the first ground truth depth map, and self-supervised training based on the estimated target depth map and the first estimated relative pose.