Method and apparatus with image processing
AssigneeSamsung Electronics Co., Ltd.
InventorsHeewon KIM, Seon Min RHEE, Jihye KIM, Seungju HAN
Life of this patent
Estimating the term…
Abstract
A method with image processing includes: generating a first surface normal image comprising surface normal vectors corresponding to pixels of a first depth image; and applying the first depth image and the first surface normal image to a first neural network, and acquiring a second depth image by changing the first depth image using the first neural network. The first neural network generates the second depth image to have an improved quality compared to the first depth image, based on an embedding vector that comprises a feature of the first depth image and a feature of the first surface normal image.
Claims26
A method with image processing, comprising:
- generating a first surface normal image comprising surface normal vectors corresponding to pixels of a first depth image;
- and
- applying the first depth image and the first surface normal image to a first neural network, and acquiring a second depth image by changing the first depth image using the first neural network,
- wherein the first neural network generates the second depth image to have an improved quality compared to the first depth image, based on an embedding vector that comprises a feature of the first depth image and a feature of the first surface normal image.
A training method, comprising:
- acquiring a second depth image of a second type by applying a first depth image of a first type and a first surface normal image corresponding to the first depth image to a first neural network;
- acquiring a third depth image of the first type by applying the acquired second depth image and a second surface normal image corresponding to the second depth image to a second neural network;
- acquiring a first generative adversarial network (GAN) loss related to whether the second depth image is an image generated by the first neural network;
- acquiring a first cycle consistency loss related to a difference between the third depth image and the first depth image;
- and
- training the first neural network based on the first GAN loss and the first cycle consistency loss.
An apparatus with image processing, comprising:
- at least one processor configured to generate a first surface normal image comprising surface normal vectors corresponding to pixels of a first depth image, apply the first depth image and the first surface normal image to a first neural network, and acquire a second depth image by changing the first depth image using the first neural network,
- wherein the first neural network is configured to generate the second depth image having an improved quality compared to the first depth image, based on an embedding vector that comprises a feature of the first depth image and a feature of the first surface normal image.
An apparatus with image processing, comprising:
- at least one processor configured to:
- acquire a second depth image having weak noise by applying a first depth image having strong noise and a first surface normal image corresponding to the first depth image to a first neural network;
- acquire a third depth image having strong noise by applying the acquired second depth image and a second surface normal image corresponding to the second depth image to a second neural network;
- acquire a first generative adversarial network (GAN) loss related to whether the second depth image is an image generated by the first neural network;
- acquire a first cycle consistency loss related to a difference between the third depth image and the first depth image;
- and
- train the first neural network based on the first GAN loss and the first cycle consistency loss.
Dependencies are inferred from the claim text.