Method and apparatus with image processing and reconstructed image generation

An apparatus, the apparatus comprising:

• one or more processors configured to:
• determine albedo data in a canonical space and depth data in the canonical space based on input image data including an object, using one or more neural network-based extraction models;
• generate deformed albedo data and deformed depth data by applying a target shape deformation value respectively to the albedo data and the depth data;
• generate resultant shaded data by performing shading based on the deformed depth data and a target illumination value;
• generate intermediate image data based on the resultant shaded data and the deformed albedo data;
• and
• generate reconstructed image data from the intermediate image data and the deformed depth data based on a target pose value.

A processor-implemented method, the method comprising:

• determining albedo data in a canonical space and depth data in the canonical space based on first training image data using a neural network-based first extraction model;
• extracting a shape deformation value, an illumination value, and a pose value from second training image data;
• generating deformed albedo data and deformed depth data by applying the shape deformation value respectively to the albedo data and the depth data;
• generating resultant shaded data by performing shading based on the deformed depth data and the illumination value;
• generating intermediate image data based on the resultant shaded data and the deformed albedo data;
• generating reconstructed image data from the intermediate image data and the deformed depth data based on the pose value;
• and
• training the one or more extraction models by updating parameters of the one or more extraction models based on the reconstructed image data and the second training image data.

A processor-implemented method, the method comprising:

• decomposing an input image into an albedo component and a depth component using a trained neural network-based extraction model;
• deforming the albedo component and the depth component based on a target shape deformation value corresponding to a local geometric change of an object of the input image;
• shading the deformed depth component based on a target illumination value;
• generating an intermediate image by combining the deformed albedo component and the shaded deformed depth component;
• and
• adjusting a pose of the intermediate image based on the deformed depth component and a target pose value.