# US 2025/0095299 A1 · Personalized deformable mesh by finetuning on personalized texture

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095299 A1
- Document ID: US20250095299A1
- Status: Published application
- Kind: A1
- Application number: 18/469,453
- Filed: 2023-09-18
- Published: 2025-03-20
- Earliest claimed: 2023-09-18
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Wei-Lun CHANG; Ke-Li CHENG; Michel Adib SARKIS
- Web page: https://patentgrep.com/patent/US20250095299A1

## CPC

- G06T 17/205: {Re-meshing}
- G06T 15/04: Texture mapping
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2219/2021: Shape modification
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Techniques and systems are provided for generating a three-dimensional facial model. For instance, a process can include: obtaining a frame, the frame including a face; obtaining a mesh 3D model of the face; generating facial texture information associated with the face based on the frame and the mesh 3D model; generating, based on the facial texture information, displacement map information; and applying the displacement map information to the mesh 3D model to generate a finetuned mesh 3D model.

## Claims

### Claim 1

A method for generating a three-dimensional facial model, the method comprising:
obtaining a frame, the frame including a face;
obtaining a mesh 3D model of the face;
generating facial texture information associated with the face based on the frame and the mesh 3D model;
generating, based on the facial texture information, displacement map information; and
applying the displacement map information to the mesh 3D model to generate a finetuned mesh 3D model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising generating UV attributes based on the mesh 3D model, wherein the facial texture information is generated based on the UV attributes.

### Claim 3 (depends on 2)

The method of claim 2, wherein the displacement map information is generated further based on the UV attributes.

### Claim 4 (depends on 2)

The method of claim 2, wherein the UV attributes are generated by a first machine learning model.

### Claim 5 (depends on 1)

The method of claim 1, wherein the facial texture information comprises a UV facial texture.

### Claim 6 (depends on 1)

The method of claim 1, wherein the facial texture information is generated by a second machine learning model, and wherein the displacement map information is generated by a third machine learning model.

### Claim 7 (depends on 6)

The method of claim 6, wherein the facial texture information comprises intermediate information from a first layer of the second machine learning model.

### Claim 8 (depends on 7)

The method of claim 7, wherein the facial texture information is generated based in part on the displacement map information, and wherein the displacement map information comprises intermediate information from a second layer of the third machine learning model.

### Claim 9 (depends on 1)

The method of claim 1, further comprising applying the facial texture information to the finetuned 3D mesh model to generate a textured 3D model.

### Claim 10 (depends on 1)

The method of claim 1, wherein applying the displacement map information to the mesh 3D model comprises:
mapping vertices of the mesh 3D model to a displacement map including the displacement map information; and
displacing the mapped vertices based on values of the displacement map.

### Claim 11

An apparatus for generating a three-dimensional facial model, the apparatus comprising:
at least one memory; and
at least one processor coupled to the at least one memory, the at least one processor being configured to:
obtain a frame, the frame including a face;
obtain a mesh 3D model of the face;
generate facial texture information associated with the face based on the frame and the mesh 3D model;
generate, based on the facial texture information, displacement map information; and
apply the displacement map information to the mesh 3D model to generate a finetuned mesh 3D model.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the at least one processor is further configured to generate UV attributes based on the mesh 3D model, wherein the facial texture information is generated based on the UV attributes.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the displacement map information is generated further based on the UV attributes.

### Claim 14 (depends on 12)

The apparatus of claim 12, wherein the UV attributes are generated by a first machine learning model.

### Claim 15 (depends on 11)

The apparatus of claim 11, wherein the facial texture information comprises a UV facial texture.

### Claim 16 (depends on 11)

The apparatus of claim 11, wherein the facial texture information is generated by a second machine learning model, and wherein the displacement map information is generated by a third machine learning model.

### Claim 17 (depends on 16)

The apparatus of claim 16, wherein the facial texture information comprises intermediate information from a first layer of the second machine learning model.

### Claim 18 (depends on 17)

The apparatus of claim 17, wherein the facial texture information is generated based in part on the displacement map information, and wherein the displacement map information comprises intermediate information from a second layer of the third machine learning model.

### Claim 19 (depends on 11)

The apparatus of claim 11, the at least one processor is further configured to apply the facial texture information to the finetuned 3D mesh model to generate a textured 3D model.

### Claim 20 (depends on 11)

The apparatus of claim 11, wherein, to apply the displacement map information to the mesh 3D model, the at least one processor is further configured to:
map vertices of the mesh 3D model to a displacement map including the displacement map information; and
displace the mapped vertices based on values of the displacement map.

### Claim 21

A non-transitory computer-readable medium having stored thereon instructions that, when executed by at least one processor, cause the at least one processor to:
obtain a frame, the frame including a face;
obtain a mesh 3D model of the face;
generate facial texture information associated with the face based on the frame and the mesh 3D model;
generate, based on the facial texture information, displacement map information; and
apply the displacement map information to the mesh 3D model to generate a finetuned mesh 3D model.

### Claim 22 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the instructions cause the at least one processor to generate UV attributes based on the mesh 3D model, wherein the facial texture information is generated based on the UV attributes.

### Claim 23 (depends on 22)

The non-transitory computer-readable medium of claim 22, wherein the displacement map information is generated further based on the UV attributes.

### Claim 24 (depends on 22)

The non-transitory computer-readable medium of claim 22, wherein the UV attributes are generated by a first machine learning model.

### Claim 25 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the facial texture information comprises a UV facial texture.

### Claim 26 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the facial texture information is generated by a second machine learning model, and wherein the displacement map information is generated by a third machine learning model.

### Claim 27 (depends on 26)

The non-transitory computer-readable medium of claim 26, wherein the facial texture information comprises intermediate information from a first layer of the second machine learning model.

### Claim 28 (depends on 27)

The non-transitory computer-readable medium of claim 27, wherein the facial texture information is generated based in part on the displacement map information, and wherein the displacement map information comprises intermediate information from a second layer of the third machine learning model.

### Claim 29 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein the instructions cause the at least one processor to apply the facial texture information to the finetuned 3D mesh model to generate a textured 3D model.

### Claim 30 (depends on 21)

The non-transitory computer-readable medium of claim 21, wherein, to apply the displacement map information to the mesh 3D model, the instructions cause the at least one processor to:
map vertices of the mesh 3D model to a displacement map including the displacement map information; and
displace the mapped vertices based on values of the displacement map.