# US 12,192,586 B2 · Method for presenting face in video call, video call apparatus, and vehicle

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,586 B2
- Document ID: US12192586B2
- Status: Granted
- Kind: B2
- Application number: 17/708,876
- Filed: 2022-03-30
- Published: 2022-07-14
- Granted: 2025-01-07
- Earliest claimed: 2019-09-30
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: Wenkang Xu; Wei Huang; Junhao Zhang; Xiaolin Huang
- Web page: https://patentgrep.com/patent/US12192586B2

## CPC

- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- G06T 7/40: Analysis of texture (depth or shape recovery from texture G06T7/529)
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 15/20: Perspective computation
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/56: relating to colour
- G06V 10/82: using neural networks
- G06V 40/166: {using acquisition arrangements}
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/174: {Facial expression recognition}
- H04N 7/142: {Constructional details of the terminal equipment, e.g. arrangements of the camera and the display}
- H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)}
- H04N 7/15: Conference systems
- H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)}
- H04N 21/41422: {located in transportation means, e.g. personal vehicle (arrangements specially adapted for transportation systems in broadcast systems H04H20/62)}
- H04N 21/4223: Cameras (H04N23/00 takes precedence)
- H04N 21/816: {involving special video data, e.g 3D video}
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30201: Face
- G06T 2219/2016: Rotation, translation, scaling
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for presenting a face in a video call includes: obtaining a key feature point of a facial expression of a user based on a face image of the user; driving a three-dimensional (3D) head image of the user by using the key feature point of the facial expression of the user, to obtain a target 3D avatar of the user, where the target 3D avatar of the user has an expression of the user; rotating the target 3D avatar based on a preset presentation angle, to obtain a target 3D avatar at the preset presentation angle; and transmitting the target 3D avatar at the preset presentation angle to a peer video call device. A user can see a 3D avatar that is of a user and that is at a preset presentation angle in real time.

## Claims

### Claim 1

A method comprising:
obtaining, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;
obtaining, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;
constructing, based on the 3D head point cloud information, a 3D head image of the user;
driving, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar has an expression of the user;
rotating, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and
sending, to a peer video call device, the second target 3D avatar.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
obtaining, based on the color face image, a face texture feature of the user; and
further constructing, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
inputting the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and
further constructing, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

### Claim 4 (depends on 3)

The method of claim 3, wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein the method further comprises:
further inputting the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and
inputting the color face image into the texture feature extraction network to obtain the face texture feature.

### Claim 5

An apparatus comprising:
a non-transitory memory configured to store instructions; and
a processor coupled to the non-transitory memory, wherein when executed by the processor, the instructions cause the apparatus to:
obtain, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;
obtain, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;
construct, based on the 3D head point cloud information, a 3D head image of the user;
drive, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar has an expression of the user;
rotate, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and
send, to a peer video call device, the second target 3D avatar.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein when executed by the processor, the instructions further cause the apparatus to:
obtain, based on the color face image, a face texture feature of the user; and
further construct, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein when executed by the processor, the instructions further cause the apparatus to:
input the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and
further construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

### Claim 8 (depends on 7)

The apparatus of claim 7, wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein when executed by the processor, the instructions further cause the apparatus to:
input the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and
input the color face image into the texture feature extraction network to obtain the face texture feature.

### Claim 9 (depends on 8)

The apparatus of claim 8, wherein the 3D head feature extraction network is a neural network that uses an encoder-decoder architecture, and wherein when executed by the processor, the instructions further cause the apparatus to:
obtain, based on the color face image and the N infrared images, N image pairs, wherein each of the N image pairs comprises the color face image and an infrared image of the user from the N infrared images; and
input the N image pairs into the neural network to obtain the 3D head point cloud information.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein when executed by the processor, the instructions further cause the apparatus to obtain, based on the N infrared images, the preset presentation angle.

### Claim 11 (depends on 5)

The apparatus of claim 5, wherein the face image is a color depth image, and wherein when executed by the processor, the instructions further cause the apparatus to:
obtain, based on the color depth image, 3D head point cloud information of the user and a face texture feature of the user; and
construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein when executed by the processor, the instructions further cause the apparatus to obtain, based on the color depth image, the preset presentation angle.

### Claim 13

A vehicle comprising:
a video call system comprising:
a processor configured to:
obtain, based on a face image of a user in a video call process, a key feature point of a facial expression of the user, wherein the face image comprises N infrared images of the user and a color face image of the user, and wherein N is an integer greater than 0;
obtain, based on the N infrared images and the color face image, a three-dimensional (3D) head point cloud information of the user;
construct, based on the 3D head point cloud information, a 3D head image of the user;
drive, using the key feature point, the 3D head image to obtain a first target 3D avatar of the user, wherein the first target 3D avatar of the user has an expression of the user;
rotate, based on a preset presentation angle, the first target 3D avatar to obtain a second target 3D avatar at the preset presentation angle; and
transmit the second target 3D avatar; and
a communications apparatus coupled to the processor and configured to:
receive the second target 3D avatar; and
send, to a peer video call device, the second target 3D avatar.

### Claim 14 (depends on 13)

The vehicle of claim 13, wherein the processor is further configured to:
obtain, based on the color face image, a face texture feature of the user; and
further construct, based on the face texture feature, the 3D head image that is a color image, wherein the 3D head point cloud information comprises first 3D head point cloud information of the user or second 3D head point cloud information of the user.

### Claim 15 (depends on 13)

The vehicle of claim 13, wherein the processor is further configured to:
input the color face image and the N infrared images into a feature extraction model to obtain 3D head point cloud information of the user and a face texture feature of the user; and
further construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

### Claim 16 (depends on 15)

The vehicle of claim 15, wherein the feature extraction model comprises a 3D head feature extraction network and a texture feature extraction network, and wherein the processor is further configured to:
input the color face image and the N infrared images into the 3D head feature extraction network to obtain the 3D head point cloud information; and
input the color face image into the texture feature extraction network to obtain the face texture feature.

### Claim 17 (depends on 16)

The vehicle of claim 16, wherein the 3D head feature extraction network is a neural network that uses an encoder-decoder architecture, and wherein the processor is further configured to:
obtain, based on the color face image and the N infrared images, N image pairs, wherein each of the N image pairs comprises the color face image and an infrared image of the user from the N infrared images; and
input the N image pairs into the neural network to obtain the 3D head point cloud information.

### Claim 18 (depends on 13)

The vehicle of claim 13, wherein the processor is further configured to obtain, based on the N infrared images, the preset presentation angle.

### Claim 19 (depends on 13)

The vehicle of claim 13, wherein the video call system further comprises an infrared camera coupled to the processor and configured to:
obtain the N infrared images of the user; and
transmit the N infrared images to the processor.

### Claim 20 (depends on 13)

The vehicle of claim 13, wherein the processor is further configured to:
obtain, based on a color depth image, 3D head point cloud information of the user and a face texture feature of the user; and
construct, based on the 3D head point cloud information and the face texture feature, the 3D head image that is a color image.

### Claim 21 (depends on 20)

The vehicle of claim 20, wherein the processor is further configured to obtain, based on the color depth image, the preset presentation angle.