# US 2025/0225777 A1 · Three-dimensional model processing method and apparatus, electronic device, and computer storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0225777 A1
- Document ID: US20250225777A1
- Status: Published application
- Kind: A1
- Application number: 19/088,699
- Filed: 2025-03-24
- Published: 2025-07-10
- Earliest claimed: 2023-04-07
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Weibin QIU
- Web page: https://patentgrep.com/patent/US20250225777A1

## CPC

- G06V 10/806: {of extracted features}
- G06T 7/33: using feature-based methods
- G06T 7/344: {involving models}
- G06T 7/75: {involving models}
- G06V 10/462: {Salient features, e.g. scale invariant feature transforms [SIFT]}
- G06V 10/82: using neural networks
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- Y02T 90/00: Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION

## Abstract

This application provides a three-dimensional model processing method performed by an electronic device. The method includes: performing key point detection on a first three-dimensional model sample, to obtain a patch index and barycentric coordinates of a patch including key points of the first three-dimensional model sample; determining three-dimensional space coordinates of each key point and displacing the three-dimensional space coordinates in the first three-dimensional model sample, to obtain a plurality of second three-dimensional model samples; combining the first and second three-dimensional model samples into a three-dimensional model sample set; and training an initialized key point detection model based on the three-dimensional model sample set, to obtain a trained key point detection model, where the trained key point detection model is configured for performing key point detection on a first to-be-registered three-dimensional model, and a key point detection result is configured for performing non-rigid deformation registration processing.

## Claims

### Claim 1

A three-dimensional model processing method, performed by an electronic device, the method comprising:
performing key point detection on a first three-dimensional model sample, to obtain a patch index and barycentric coordinates of a patch in which each of a plurality of key points of the first three-dimensional model sample is located;
determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point;
displacing the three-dimensional space coordinates of the plurality of key points in the first three-dimensional model sample, to obtain a plurality of second three-dimensional model samples;
combining the plurality of second three-dimensional model samples and the first three-dimensional model sample into a three-dimensional model sample set; and
training an initialized key point detection model based on the three-dimensional model sample set, to obtain a trained key point detection model, wherein the trained key point detection model is configured for performing key point detection on a first target three-dimensional model.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the performing key point detection on the first three-dimensional model sample further comprises:
obtaining a plurality of depth maps of the first three-dimensional model sample, each depth map being obtained from a different direction of the first three-dimensional model sample;
performing feature extraction processing on each of the plurality of depth maps, to obtain a feature map of the first three-dimensional model sample for the depth map;
combining the feature maps, to obtain a concatenated feature;
activating the concatenated feature, to obtain current coordinates of each key point in the first three-dimensional model sample; and
determining, based on the current coordinates of each key point, a patch in which each key point is located and barycentric coordinates corresponding to the key point.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point comprises:
querying, based on the patch index, vertex coordinates of each vertex of a patch in which the key point is located;
determining a correspondence between each barycentric coordinate value in the barycentric coordinates and each vertex; and
performing weighted summation on each vertex coordinate and the corresponding barycentric coordinate value, to obtain the three-dimensional space coordinates of the key point.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the method further comprises:
performing key point detection on the first target three-dimensional model using the trained key point detection model, to obtain key point information of the first target three-dimensional model, the key point information comprising: a patch index of a patch in which each key point is located, and barycentric coordinates of the patch;
obtaining key point information of a first reference three-dimensional model; and
performing non-rigid deformation registration on the key point information of the first target three-dimensional model by using the key point information of the first reference three-dimensional model as a registration reference object, to obtain a registered first target three-dimensional model.

### Claim 5 (depends on 4)

The method according to claim 4, wherein the method further comprises:
using the registered first target three-dimensional model as a first registered three-dimensional model;
adjusting positions of the first registered three-dimensional model and the first reference three-dimensional model, to enable the first registered three-dimensional model and the first reference three-dimensional model to be in an overlapping state;
obtaining a plurality of reference points on a surface of the first registered three-dimensional model and a shortest distance between each reference point and the first reference three-dimensional model; and
selecting an average value of the shortest distances between the reference points and the first reference three-dimensional model as a registration error between the first registered three-dimensional model and the first reference three-dimensional model.

### Claim 6 (depends on 5)

The method according to claim 5, wherein the method further comprises:
obtaining a registration error of non-rigid deformation registration performed for a plurality of first target three-dimensional models;
sorting the first target three-dimensional models in a descending order based on the registration error of each first target three-dimensional model, to obtain a descending sorting list;
obtaining corresponding key point information for a preset quantity of first target three-dimensional models starting from the head in the descending sorting list;
combining the preset quantity of first target three-dimensional models and the corresponding key point information into a three-dimensional model sample set; and
iteratively training the trained key point detection model based on the three-dimensional model sample set, to obtain an adjusted key point detection model.

### Claim 7 (depends on 1)

The method according to claim 1, wherein the displacing the three-dimensional space coordinates of the plurality of key points in the first three-dimensional model sample comprises:
dividing the key points into a plurality of key point groups according to a position of each key point in the first three-dimensional model sample; and
repeatedly performing the following processing for each key point group:
using at least one key point in the key point group as a target key point, and using a geometric center of each target key point as an origin of a local coordinate system of each target key point;
performing position movement on each target key point in the local coordinate system of each target key point, to obtain a position of a moved target key point;
mapping the position of each moved target key point from the local coordinate system to a global coordinate system corresponding to the first three-dimensional model sample, to obtain a position of each moved target key point in the global coordinate system; and
replacing a position of each unmoved target key point in the first three-dimensional model sample with the position of each moved target key point, to obtain a second three-dimensional model sample.

### Claim 8 (depends on 1)

The method according to claim 1, wherein the training an initialized key point detection model based on the three-dimensional model sample set comprises:
performing key point detection on the three-dimensional model sample set using the initialized key point detection model, to obtain predicted three-dimensional space coordinates of predicted key points of each second three-dimensional model sample and the first three-dimensional model sample;
determining a difference between each predicted three-dimensional space coordinate and the three-dimensional space coordinates of each key point; and
determining a training loss of the initialized key point detection model based on the difference, and updating a parameter of the initialized key point detection model based on the training loss, to obtain the trained key point detection model.

### Claim 9

An electronic device, comprising:
a memory, configured to store computer-executable instructions; and
a processor, configured to implement, when executing the computer-executable instructions stored in the memory, a three-dimensional model processing method including:
performing key point detection on a first three-dimensional model sample, to obtain a patch index and barycentric coordinates of a patch in which each of a plurality of key points of the first three-dimensional model sample is located;
determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point;
displacing the three-dimensional space coordinates of the plurality of key points in the first three-dimensional model sample, to obtain a plurality of second three-dimensional model samples;
combining the plurality of second three-dimensional model samples and the first three-dimensional model sample into a three-dimensional model sample set; and
training an initialized key point detection model based on the three-dimensional model sample set, to obtain a trained key point detection model, wherein the trained key point detection model is configured for performing key point detection on a first target three-dimensional model.

### Claim 10 (depends on 9)

The electronic device according to claim 9, wherein the performing key point detection on the first three-dimensional model sample further comprises:
obtaining a plurality of depth maps of the first three-dimensional model sample, each depth map being obtained from a different direction of the first three-dimensional model sample;
performing feature extraction processing on each of the plurality of depth maps, to obtain a feature map of the first three-dimensional model sample for the depth map;
combining the feature maps, to obtain a concatenated feature;
activating the concatenated feature, to obtain current coordinates of each key point in the first three-dimensional model sample; and
determining, based on the current coordinates of each key point, a patch in which each key point is located and barycentric coordinates corresponding to the key point.

### Claim 11 (depends on 9)

The electronic device according to claim 9, wherein the determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point comprises:
querying, based on the patch index, vertex coordinates of each vertex of a patch in which the key point is located;
determining a correspondence between each barycentric coordinate value in the barycentric coordinates and each vertex; and
performing weighted summation on each vertex coordinate and the corresponding barycentric coordinate value, to obtain the three-dimensional space coordinates of the key point.

### Claim 12 (depends on 9)

The electronic device according to claim 9, wherein the method further comprises:
performing key point detection on the first target three-dimensional model using the trained key point detection model, to obtain key point information of the first target three-dimensional model, the key point information comprising: a patch index of a patch in which each key point is located, and barycentric coordinates of the patch;
obtaining key point information of a first reference three-dimensional model; and
performing non-rigid deformation registration on the key point information of the first target three-dimensional model by using the key point information of the first reference three-dimensional model as a registration reference object, to obtain a registered first target three-dimensional model.

### Claim 13 (depends on 12)

The electronic device according to claim 12, wherein the method further comprises:
using the registered first target three-dimensional model as a first registered three-dimensional model;
adjusting positions of the first registered three-dimensional model and the first reference three-dimensional model, to enable the first registered three-dimensional model and the first reference three-dimensional model to be in an overlapping state;
obtaining a plurality of reference points on a surface of the first registered three-dimensional model and a shortest distance between each reference point and the first reference three-dimensional model; and
selecting an average value of the shortest distances between the reference points and the first reference three-dimensional model as a registration error between the first registered three-dimensional model and the first reference three-dimensional model.

### Claim 14 (depends on 13)

The electronic device according to claim 13, wherein the method further comprises:
obtaining a registration error of non-rigid deformation registration performed for a plurality of first target three-dimensional models;
sorting the first target three-dimensional models in a descending order based on the registration error of each first target three-dimensional model, to obtain a descending sorting list;
obtaining corresponding key point information for a preset quantity of first target three-dimensional models starting from the head in the descending sorting list;
combining the preset quantity of first target three-dimensional models and the corresponding key point information into a three-dimensional model sample set; and
iteratively training the trained key point detection model based on the three-dimensional model sample set, to obtain an adjusted key point detection model.

### Claim 15 (depends on 9)

The electronic device according to claim 9, wherein the displacing the three-dimensional space coordinates of the plurality of key points in the first three-dimensional model sample comprises:
dividing the key points into a plurality of key point groups according to a position of each key point in the first three-dimensional model sample; and
repeatedly performing the following processing for each key point group:
using at least one key point in the key point group as a target key point, and using a geometric center of each target key point as an origin of a local coordinate system of each target key point;
performing position movement on each target key point in the local coordinate system of each target key point, to obtain a position of a moved target key point;
mapping the position of each moved target key point from the local coordinate system to a global coordinate system corresponding to the first three-dimensional model sample, to obtain a position of each moved target key point in the global coordinate system; and
replacing a position of each unmoved target key point in the first three-dimensional model sample with the position of each moved target key point, to obtain a second three-dimensional model sample.

### Claim 16 (depends on 9)

The electronic device according to claim 9, wherein the training an initialized key point detection model based on the three-dimensional model sample set comprises:
performing key point detection on the three-dimensional model sample set using the initialized key point detection model, to obtain predicted three-dimensional space coordinates of predicted key points of each second three-dimensional model sample and the first three-dimensional model sample;
determining a difference between each predicted three-dimensional space coordinate and the three-dimensional space coordinates of each key point; and
determining a training loss of the initialized key point detection model based on the difference, and updating a parameter of the initialized key point detection model based on the training loss, to obtain the trained key point detection model.

### Claim 17

A non-transitory computer-readable storage medium, storing computer-executable instructions, the computer-executable instructions, when executed by a processor of an electronic device, causing the electronic device to implement a three-dimensional model processing method including:
performing key point detection on a first three-dimensional model sample, to obtain a patch index and barycentric coordinates of a patch in which each of a plurality of key points of the first three-dimensional model sample is located;
determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point;
displacing the three-dimensional space coordinates of the plurality of key points in the first three-dimensional model sample, to obtain a plurality of second three-dimensional model samples;
combining the plurality of second three-dimensional model samples and the first three-dimensional model sample into a three-dimensional model sample set; and
training an initialized key point detection model based on the three-dimensional model sample set, to obtain a trained key point detection model, wherein the trained key point detection model is configured for performing key point detection on a first target three-dimensional model.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the performing key point detection on the first three-dimensional model sample further comprises:
obtaining a plurality of depth maps of the first three-dimensional model sample, each depth map being obtained from a different direction of the first three-dimensional model sample;
performing feature extraction processing on each of the plurality of depth maps, to obtain a feature map of the first three-dimensional model sample for the depth map;
combining the feature maps, to obtain a concatenated feature;
activating the concatenated feature, to obtain current coordinates of each key point in the first three-dimensional model sample; and
determining, based on the current coordinates of each key point, a patch in which each key point is located and barycentric coordinates corresponding to the key point.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the determining three-dimensional space coordinates of each of the plurality of key points based on the patch index and the barycentric coordinates of the key point comprises:
querying, based on the patch index, vertex coordinates of each vertex of a patch in which the key point is located;
determining a correspondence between each barycentric coordinate value in the barycentric coordinates and each vertex; and
performing weighted summation on each vertex coordinate and the corresponding barycentric coordinate value, to obtain the three-dimensional space coordinates of the key point.

### Claim 20 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein the method further comprises:
performing key point detection on the first target three-dimensional model using the trained key point detection model, to obtain key point information of the first target three-dimensional model, the key point information comprising: a patch index of a patch in which each key point is located, and barycentric coordinates of the patch;
obtaining key point information of a first reference three-dimensional model; and
performing non-rigid deformation registration on the key point information of the first target three-dimensional model by using the key point information of the first reference three-dimensional model as a registration reference object, to obtain a registered first target three-dimensional model.
