Method and apparatus to complement depth image
AssigneeSamsung Electronics Co., Ltd.
InventorsZhaohui Lv, Weijia Zhang, Mingming Fan
Life of this patent
Estimating the term…
Abstract
A method and apparatus for complementing a depth image are provided. The method includes obtaining a color image and a corresponding depth image, obtaining a first depth image based on the color image, using a first deep neural network (DNN), obtaining a second depth image based on the depth image and an intermediate feature image generated by each intermediate layer of the first DNN, using a second DNN, and obtaining a final depth image by merging the first depth image and the second depth image.
Claims27
A processor-implemented method of complementing a depth image, comprising:
- obtaining a first depth image based on only a color image, using a first deep neural network (DNN);
- obtaining a second depth image based on only an original depth image, corresponding to the color image, and an intermediate feature image generated by each intermediate layer of the first DNN, using a second DNN to perform feature fusion including inputting the intermediate feature image from each intermediate layer of the first DNN to each corresponding intermediate layer of the second DNN;
- and
- obtaining a final depth image by merging the first depth image and the second depth image,
- wherein the feature fusion is only performed at the second DNN.
An apparatus for complementing a depth image, comprising:
- one or more processors configured to:
- obtain a first depth image based on only a color image, using a first deep neural network (DNN);
- obtain a second depth image based on only an original depth image, corresponding to the color image, and an intermediate feature image generated by each intermediate layer of the first DNN, using a second DNN to perform feature fusion including inputting the intermediate feature image from each intermediate layer of the first DNN to each corresponding intermediate layer of the second DNN;
- and
- obtain a final depth image by merging the first depth image and the second depth image,
- wherein the feature fusion is only performed at the second DNN.
Dependencies are inferred from the claim text.