# US 12,183,036 B2 · Image processing method and apparatus, electronic device, and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,183,036 B2
- Document ID: US12183036B2
- Status: Granted
- Kind: B2
- Application number: 17/706,242
- Filed: 2022-03-28
- Published: 2022-07-07
- Granted: 2024-12-31
- Earliest claimed: 2020-01-14
- Assignees: Tencent Technology (Shenzhen) Company Limited
- Applicants: Tencent Technology (Shenzhen) Company Limited
- Inventors: Jie Liu
- Web page: https://patentgrep.com/patent/US12183036B2

## CPC

- G06T 7/75: {involving models}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- A63F 13/35: Details of game servers
- A63F 13/56: Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
- A63F 13/60: Generating or modifying game content before or while executing the game program, e.g. authoring tools specially adapted for game development or game-integrated level editor
- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- A63F 2300/6607: for animating game characters, e.g. skeleton kinematics
- G06T 2207/30196: Human being; Person
- G06T 2219/2021: Shape modification
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

This disclosure provides an image processing method and apparatus, an electronic device, and a storage medium. The method includes: obtaining, by an electronic device, a reference surface model of a first skeleton posture of a virtual object; obtaining one or more second skeleton postures of the virtual object; generating an one or more exemplary surface models of the one or more second skeleton postures based on the reference surface model and the one or more second skeleton postures; determining a posture transformation matrix between the first skeleton posture and the second skeleton posture; and converting the exemplary surface model and the reference surface model to a same coordinate space based on the posture transformation matrix, to generate the virtual object according to the reference surface model and the one or more exemplary surface models in the coordinate space.

## Claims

### Claim 1

An image processing method, performed by an electronic device, the method comprising:
obtaining at least one reference surface model of a first skeleton posture of a virtual object, the at least one reference model located in a reference coordinate space;
obtaining one or more second skeleton postures of the virtual object;
generating one or more exemplary surface models of the one or more second skeleton postures based on the at least one reference surface model and the one or more second skeleton postures;
determining a posture transformation matrix between the first skeleton posture and the one or more second skeleton postures;
converting the one or more exemplary surface models and the at least one reference surface model to the reference coordinate space using the posture transformation matrix to generate the virtual object according to the at least one reference surface model and the one or more exemplary surface models in the reference coordinate space, wherein converting the one or more exemplary surface models and the at least one reference surface model to the reference coordinate space using the posture transformation matrix comprises:
performing a coordinate transformation on the one or more exemplary surface models using the posture transformation matrix to convert the one or more exemplary surface models to the reference coordinate space in which the at least one reference surface model is located;
obtaining a target skeleton posture of the virtual object;
performing skin processing on the at least one reference surface model based on the target skeleton posture to obtain a target skeleton-skinning model of the target skeleton posture;
determining a modified surface model of the target skeleton posture based on the at least one reference surface model and the one or more exemplary surface models in the reference coordinate space; and
generating the virtual object based on the target skeleton-skinning model and the modified surface model.

### Claim 2 (depends on 1)

The method according to claim 1, wherein generating the one or more exemplary surface models of the one or more second skeleton postures based on the at least one reference surface model and the one or more second skeleton postures comprises:
performing skin processing on the at least one reference surface model based on the one or more second skeleton postures to obtain an exemplary skeleton-skinning model of the one or more second skeleton postures;
extracting surface data of the exemplary skeleton-skinning model; and
generating the one or more exemplary surface models of the one or more second skeleton postures according to the surface data.

### Claim 3 (depends on 2)

The method according to claim 2, wherein before extracting the surface data of the exemplary skeleton-skinning model, the method further comprises:
obtaining a modification request for the exemplary skeleton-skinning model from a requester; and
opening a modification permission of the exemplary skeleton-skinning model to the requester after a verification on the modification request succeeds to allow the requester to modify the exemplary skeleton-skinning model; and
extracting the surface data of the exemplary skeleton-skinning model comprises extracting the surface data of the modified exemplary skeleton-skinning model.

### Claim 4 (depends on 1)

The method according to claim 1, wherein determining the modified surface model of the target skeleton posture based on the at least one reference surface model and the one or more exemplary surface models in the reference coordinate space comprises:
determining, when the first skeleton posture and the one or more second skeleton postures are fused into the target skeleton posture, a weight of the first skeleton posture and a weight of the one or more second skeleton postures; and
fusing the at least one reference surface model and the one or more exemplary surface models based on the weight of the first skeleton posture and the weight of the one or more second skeleton postures to obtain the modified surface model of the target skeleton posture.

### Claim 5 (depends on 4)

The method according to claim 4, wherein determining, when the first skeleton posture and the one or more second skeleton postures are fused into the target skeleton posture, the weight of the first skeleton posture and the weight of the one or more second skeleton postures comprises:
when the first skeleton posture and the one or more second skeleton posture are fused into the target skeleton posture, performing interpolation processing between the first skeleton posture and the one or more second skeleton postures by using a radial basis function to obtain the weight of the first skeleton posture and the weight of the one or more second skeleton postures.

### Claim 6 (depends on 4)

The method according to claim 4, wherein fusing the at least one reference surface model and the one or more exemplary surface models based on the weight of the first skeleton posture and the weight of the one or more second skeleton postures to obtain the modified surface model of the target skeleton posture comprises:
performing weighting processing on the at least one reference surface model and the one or more exemplary surface models by using the weight of the first skeleton posture as a weight of the at least one reference surface model and using the weight of the one or more second skeleton postures as a weight of the one or more exemplary surface models to obtain the modified surface model of the target skeleton posture.

### Claim 7 (depends on 1)

The method according to claim 1, wherein generating the virtual object based on the target skeleton-skinning model and the modified surface model comprises:
determining a posture transformation matrix between the first skeleton posture and the target skeleton posture as a target posture transformation matrix;
performing a coordinate transformation on the modified surface model to change from the reference coordinate space to a coordinate space of the target skeleton posture based on the target posture transformation matrix to obtain the modified surface model in the coordinate space of the target skeleton posture; and
performing surface deformation processing on the target skeleton-skinning model; and
after performing the coordinate transformation on the modified surface model, superimposing the modified surface model and the target skeleton-skinning model, to generate the virtual object.

### Claim 8 (depends on 7)

The method according to claim 7, wherein before superimposing the modified surface model and the target skeleton-skinning model, the method further comprises performing surface deformation processing on the target skeleton-skinning model; and
wherein superimposing the modified surface model and the target skeleton-skinning model to generate the virtual object comprises superimposing the modified surface model and the target skeleton-skinning model obtained after the surface deformation processing to generate the virtual object.

### Claim 9 (depends on 1)

The method according to claim 1, wherein obtaining the target skeleton posture of the virtual object comprises:
obtaining manipulation data for the virtual object; and
determining the target skeleton posture of the virtual object based on the manipulation data.

### Claim 10 (depends on 1)

The method according to claim 1, wherein obtaining the target skeleton posture of the virtual object comprises:
obtaining sensor data;
determining a user action based on the sensor data; and
determining the target skeleton posture of the virtual object based on the user action.

### Claim 11 (depends on 1)

The method according to claim 1, wherein
the one or more second skeleton postures comprises two or more second skeleton postures;
generating the one or more exemplary surface models of the one or more second skeleton postures based on the at least one reference surface model and the one or more second skeleton postures comprises:
respectively generating two or more exemplary surface models of the two or more second skeleton postures based on the at least one reference surface model and the two or more second skeleton postures.

### Claim 12 (depends on 11)

The method according to claim 11, wherein
determining the posture transformation matrix between the first skeleton posture and the one or more second skeleton postures comprises:
determining a posture transformation matrix between the first skeleton posture and each of the two or more second skeleton postures; and
converting the one or more exemplary surface models and the at least one reference surface model to the reference coordinate space based on the posture transformation matrix comprises:
converting the two or more exemplary surface models and the at least one reference surface model to the reference coordinate space based on the posture transformation matrix between the first skeleton posture and each of the two or more second skeleton postures.

### Claim 13 (depends on 12)

The method according to claim 12, further comprising:
obtaining a target skeleton posture of the virtual object;
performing skin processing on the at least one reference surface model based on the target skeleton posture to obtain a target skeleton-skinning model of the target skeleton posture;
determining a modified surface model of the target skeleton posture based on the at least one reference surface model and the two or more exemplary surface models in the reference coordinate space; and
generating the virtual object based on the target skeleton-skinning model and the modified surface model.

### Claim 14 (depends on 1)

The method according to claim 1, further comprising:
rendering the generated virtual object, to display the virtual object in a virtual scene.

### Claim 15

An electronic device, comprising:
a memory storing computer-readable instructions; and
a processor, configured to execute the computer-readable instructions stored in the memory, to cause the electronic device to:
obtain at least one reference surface model of a first skeleton posture of a virtual object, the at least one reference model located in a reference coordinate space;
obtain one or more second skeleton postures of the virtual object;
generate one or more exemplary surface models of the one or more second skeleton postures based on the at least one reference surface model and the one or more second skeleton postures;
determine a posture transformation matrix between the first skeleton posture and the one or more second skeleton postures;
convert the one or more exemplary surface models and the at least one reference surface model to the reference coordinate space using the posture transformation matrix to generate the virtual object according to the at least one reference surface model and the one or more exemplary surface models in the reference coordinate space, wherein in order to convert the one or more exemplary surface models and the at least one reference surface model to the reference coordinate space using the posture transformation matrix, the processor, upon execution of the computer-readable instructions, is configured to:
perform a coordinate transformation on the one or more exemplary surface models using the posture transformation matrix to convert the one or more exemplary surface models to the reference coordinate space in which the at least one reference surface model is located;
obtain a target skeleton posture of the virtual object;
perform skin processing on the at least one reference surface model based on the target skeleton posture to obtain a target skeleton-skinning model of the target skeleton posture;
determine a modified surface model of the target skeleton posture based on the at least one reference surface model and the one or more exemplary surface models in the reference coordinate space; and
generate the virtual object based on the target skeleton-skinning model and the modified surface model.

### Claim 16 (depends on 15)

The electronic device of claim 15, wherein in order to generate the one or more exemplary surface models of the one or more second skeleton postures based on the at least one reference surface model and the one or more second skeleton postures, the processor, upon execution of the computer-readable instructions, is configured to:
perform skin processing on the at least one reference surface model based on the one or more second skeleton postures to obtain an exemplary skeleton-skinning model of the one or more second skeleton postures;
extract surface data of the exemplary skeleton-skinning model; and
generate the one or more exemplary surface models of the one or more second skeleton postures according to the surface data.