# US 2025/0148716 A1 · Method for 3-dimension model reconstruction based on multi-view images and apparatus for the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148716 A1
- Document ID: US20250148716A1
- Status: Published application
- Kind: A1
- Application number: 18/934,551
- Filed: 2024-11-01
- Published: 2025-05-08
- Earliest claimed: 2023-11-03
- Assignees: Electronics and Telecommunications Research Institute; Kwangwoon University Industry-Academic Collaboration Foundation
- Applicants: Electronics and Telecommunications Research Institute; Kwangwoon University Industry-Academic Collaboration Foundation
- Inventors: Hee Kyung LEE; Gi Mun UM; Ju Yong CHANG; Seung Young NOH; Jin Gyu HONG
- Web page: https://patentgrep.com/patent/US20250148716A1

## CPC

- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 7/344: {involving models}
- G06T 7/593: from stereo images
- G06T 15/04: Texture mapping
- G06T 2200/04: involving 3D image data
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

The present disclosure relates to a method for reconstructing a three-dimensional model based on a multi-view image and a device therefor. A method for reconstructing a three-dimensional model according to an embodiment of the present disclosure may include obtaining n (n>1) multi-view images, and a first two-dimensional feature map and a first three-dimensional feature map, based on n depth maps for the n multi-view images; estimating a mesh through occupancy prediction based on the first two-dimensional feature map and the first three-dimensional feature map; and applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model.

## Claims

### Claim 1

A method for reconstructing a three-dimensional model, the method comprising:
obtaining n (n>1) multi-view images, and a first two-dimensional feature map and a first three-dimensional feature map, based on and n depth maps for the n multi-view images;
estimating a mesh through an occupancy prediction based on the first two-dimensional feature map and the first three-dimensional feature map; and
applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the first two-dimensional feature map corresponds to n first improved two-dimensional feature maps based on n first pixel-aligned two-dimensional feature maps extracted based on the n multi-view images and the n depth maps.

### Claim 3 (depends on 2)

The method of claim 2, wherein:
the n first improved two-dimensional feature maps is obtained through a feature-improved multi-layer perceptron for a i-th view, for each i (1≤i≤n)-th first pixel-aligned two-dimensional feature map of the n first pixel-aligned two-dimensional feature maps.

### Claim 4 (depends on 3)

The method of claim 3, wherein:
an input of the feature-improved multi-layer perceptron for the i-th view corresponds to a bilinear interpolation result for:
a i-th first pixel-aligned two-dimensional feature map; and
a two-dimensional projection π(x) corresponding to a i-th view for a three-dimensional query coordinate x,
a final dimension of each of the n first improved two-dimensional feature maps is determined equally as 256/n.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
the first three-dimensional feature map corresponds to:
a three-dimensional volume estimated through a depth map backprojection based on the n multi-view images and the n depth maps; and
a first voxel-aligned three-dimensional feature map transformed through a first three-dimensional feature extraction for the estimated three-dimensional volume.

### Claim 6 (depends on 5)

The method of claim 5, wherein:
the depth map backprojection includes:
performing a deconvolution on an image feature map based on the n multi-view images and the n depth maps;
obtaining a three-dimensional image feature map through a repetition and a concatenation for an image feature map to which the deconvolution is applied; and
estimating the three-dimensional volume through a three-dimensional convolution for the three-dimensional image feature map.

### Claim 7 (depends on 5)

The method of claim 5, wherein:
a predetermined number of points are extracted from the estimated three-dimensional volume, and the first voxel-aligned three-dimensional feature map is obtained based on the predetermined number of points.

### Claim 8 (depends on 1)

The method of claim 1, wherein:
the occupancy prediction includes outputting a value estimating whether a three-dimensional query coordinate x is inside or outside the three-dimensional model through an occupancy prediction multi-layer perceptron based on:
a two-dimensional feature map for an occupancy prediction in which the n first improved two-dimensional feature maps are fused; and
the first three-dimensional feature map.

### Claim 9 (depends on 1)

The method of claim 1, wherein:
the first two-dimensional feature map includes a geometric feature for the n multi-view images and the n depth maps,
the second two-dimensional feature map includes a texture-related feature for the n multi-view images and the n depth maps,
the first three-dimensional feature map is obtained based on a textureless three-dimensional volume,
the second three-dimensional feature map is obtained based on a textured three-dimensional volume.

### Claim 10 (depends on 1)

The method of claim 1, wherein:
the n depth maps are estimated based on:
the n multi-view images; and
a mesh estimated based on a first image among the n multi-view images.

### Claim 11

A method for reconstructing a three-dimensional model, the method comprising:
obtaining n (n>1) multi-view images, and a second two-dimensional feature map and a second three-dimensional feature map, based on and n depth maps for the n multi-view images;
estimating a texture through a texture prediction based on the second two-dimensional feature map and the second three-dimensional feature map; and
applying the estimated texture to a mesh estimated based on a first two-dimensional feature map and a first three-dimensional feature map obtained based on the n multi-view images and the n depth maps to obtain a texture-applied final model.

### Claim 12 (depends on 11)

The method of claim 11, wherein:
the second two-dimensional feature map corresponds to n improved two-dimensional feature maps based on n second pixel-aligned two-dimensional feature maps extracted based on the n multi-view images and the n depth maps.

### Claim 13 (depends on 12)

The method of claim 12, wherein:
the n second improved two-dimensional feature maps are obtained through a feature-improved multi-layer perceptron for a i-th view for each i (1≤i≤n)-th second pixel-aligned two-dimensional feature map of the n second pixel-aligned two-dimensional feature maps.

### Claim 14 (depends on 13)

The method of claim 13, wherein:
an input of the feature-improved multi-layer perceptron for the i-th view corresponds to a bilinear interpolation result for:
a i-th second pixel-aligned two-dimensional feature map; and
a two-dimensional projection π(x) corresponding to a i-th view for a three-dimensional query coordinate x,
a final dimension of each of the n second improved two-dimensional feature maps is determined equally as 256/n.

### Claim 15 (depends on 11)

The method of claim 11, wherein:
the second three-dimensional feature map corresponds to:
a color three-dimensional volume estimated through a color image backprojection based on the n multi-view images and the n depth maps; and
a second voxel-aligned three-dimensional feature map transformed through a second three-dimensional feature extraction for the estimated color three-dimensional volume.

### Claim 16 (depends on 12)

The method of claim 12, wherein:
a two-dimensional feature map for a texture prediction is obtained by fusing:
a feature map in which the n second improved two-dimensional feature maps are fused; and
a two-dimensional feature map for an occupancy prediction in which n first improved two-dimensional feature maps are fused.

### Claim 17 (depends on 16)

The method of claim 16, wherein:
the texture prediction includes obtaining a predicted red green blue (RGB) value for a three-dimensional query coordinate x, a first combining parameter λ, and a union of n second combining parameters ω1, ω2, . . . , ωn through a texture prediction multi-layer perceptron based on:
the two-dimensional feature map for the texture prediction; and
the second three-dimensional feature map.

### Claim 18 (depends on 17)

The method of claim 17, wherein:
the estimated texture is obtained, through the first combining parameter A, by linearly combining:
the predicted RGB value; and
an observed RGB value obtained based on the n multi-view images and the n second combining parameters ω1, ω2, . . . , ωn.

### Claim 19

A device for reconstructing a three-dimensional model, the device comprising:
at least one processor; and
at least one memory operably connected to the at least one processor, and storing an instruction to make the device perform an operation when executed by the at least one processor,
wherein the operation includes:
obtaining n (n>1) multi-view images, and a first two-dimensional feature map and a first three-dimensional feature map, based on n depth maps for the n multi-view images;
estimating a mesh through an occupancy prediction based on the first two-dimensional feature map and the first three-dimensional feature map; and
applying a texture estimated based on a second two-dimensional feature map and a second three-dimensional feature map obtained based on the n multi-view images and the n depth maps to the estimated mesh to obtain a texture-applied final model.
