Method and apparatus with feature embedding
AssigneeSamsung Electronics Co., Ltd.
InventorsSeokhwan JANG, Minjung SON, Hyun Sung CHANG
Life of this patent
Estimating the term…
Abstract
A method and apparatus with feature embedding is provided. The method includes estimating a depth map for each of plural two-dimensional (2D) input images, transforming the depth maps into three-dimensional (3D) information in a point cloud form based on an aggregation of the depth maps, and generating an embedded feature by applying the 3D information to a machine learning model, where the embedded feature includes information about a 3D shape corresponding to a 2D object in the plural 2D input images.
Claims36
A processor-implemented method, the method comprising:
- estimating a depth map for each of plural two-dimensional (2D) input images;
- transforming the depth maps into three-dimensional (3D) information in a point cloud form based on an aggregation of the depth maps;
- and
- generating an embedded feature by applying the 3D information to a machine learning model,
- wherein the embedded feature includes information about a 3D shape corresponding to a 2D object in the plural 2D input images.
An apparatus, the apparatus comprising:
- a processor configured to:
- estimate a depth map for each of plural two-dimensional (2D) input images;
- transform the depth maps into three-dimensional (3D) information in a point cloud form based on an aggregation of the depth maps;
- and
- generate an embedded feature by applying the 3D information to a machine learning model,
- wherein the embedded feature includes information about a 3D shape corresponding to a 2D object included in the plural 2D input images.
An augmented reality (AR) apparatus comprising:
- a communication interface configured to receive plural two-dimensional (2D) input images comprising multi-view images or a sequence image;
- a processor configured to:
- estimate respective depth maps for the plural 2D input images;
- transform the respective depth maps into three-dimensional (3D) information in a point cloud form based on an aggregation of the respective depth maps;
- perform encoding of a feature to include information about a 3D shape corresponding to a 2D object included in the plural 2D input images by applying the 3D information to a neural network;
- and
- reconstruct the 3D shape corresponding to the 2D object based on the feature;
- and
- a display configured to display an output image comprising the 3D shape.
An apparatus, the apparatus comprising:
- a memory storing an embedded feature that includes information about a 3D shape corresponding to a 2D object;
- a display;
- and
- a processor configured to reconstruct the 2D object as a 3D object based on the embedded feature and control display of the reconstructed 2D object,
- wherein the embedded feature is a feature reflecting having been generated based on depth maps for each of plural two-dimensional (2D) images, a transformation of the depth maps into three-dimensional (3D) information based on an aggregation of the depth maps, and by an application of the 3D information to a neural network.
Dependencies are inferred from the claim text.