# US 2023/0267584 A1 · Virtual clothing changing method and apparatus, and device and medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0267584 A1
- Document ID: US20230267584A1
- Status: Published application
- Kind: A1
- Application number: 18/012,128
- Filed: 2021-05-26
- Published: 2023-08-24
- Earliest claimed: 2020-06-24
- Assignees: Beijing Bytedance Network Technology Co., Ltd.
- Applicants: Beijing Bytedance Network Technology Co., Ltd.
- Inventors: Yifan LU
- Web page: https://patentgrep.com/patent/US20230267584A1

## CPC

- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- G06Q 30/015: Providing customer assistance, e.g. assisting a customer within a business location or via helpdesk
- G06Q 30/0623: {by investigating goods or services}
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20132: Image cropping
- G06T 2207/20221: Image fusion; Image merging
- G06T 2219/2016: Rotation, translation, scaling
- G06T 2219/2024: Style variation
- G06V 2201/07: Target detection
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed are a virtual clothing changing method and apparatus. One specific embodiment of the method comprises: extracting foot images from a target image in which the feet are displayed; generating foot posture information on the basis of the foot images; and on the basis of the foot posture information, superimposing a clothing pattern onto the feet displayed in the target image. The embodiment not only makes it convenient for a customer to buy suitable clothing, but also ensures the accuracy of virtual clothing changing, thereby facilitating an improvement in the experience of the customer buying the clothing.

## Claims

### Claim 1

A method for simulating clothing changing, comprising:
extracting a foot image from a target image displaying a foot;
generating foot posture information based on the foot image; and
superimposing, based on the foot posture information, a pattern of a clothing on the foot displayed in the target image.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the extracting a foot image from a target image displaying a foot comprises:
inputting the target image into a pre-trained target detection network to obtain a first rectangular box;
expanding, based on a center of the first rectangular box, the first rectangular box to obtain a second rectangular box; and
clipping, from the target image, an image corresponding to the second rectangular box as the foot image.

### Claim 3 (depends on 1)

The method according to claim 1, wherein the generating foot posture information based on the foot image comprises:
inputting the foot image into a pre-trained foot pose estimation network to obtain foot pose information, wherein the foot pose information comprises: a quaternion of a position of the foot in the foot image relative to a position of an initial three-dimensional foot model, translation information of the position of the foot in the foot image relative to the position of the initial three-dimensional foot model, and left-right foot information in the foot image.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the superimposing, based on the foot posture information, a pattern of a clothing on the foot displayed in the target image comprises:
acquiring a three-dimensional clothing model matching the initial three-dimensional foot model;
performing, based on the foot posture information, three-dimensional transformation on the three-dimensional clothing model by using a 3D engine, to superimpose an image of the three-dimensional clothing model on an image at the position of the foot displayed in the foot image, to obtain a superimposed foot image; and
obtaining a rendered target image based on the superimposed foot image.

### Claim 5 (depends on 2)

The method according to claim 2, wherein the target detection network is one of the following:
a Fast R-CNN algorithm network, a Fast R-CNN algorithm network, a YOLO series algorithm network, and an SSD algorithm network.

### Claim 6

A device for simulating clothing changing, comprising:
at least one processor; and
at least one memory communicatively coupled to the at least one processor and storing instructions that upon execution by the at least one processor cause the apparatus to:
extract a foot image from a target image displaying a foot;
generate foot posture information based on the foot image; and
superimpose, based on the foot posture information, a pattern of a clothing on the foot displayed in the target image.

### Claim 7

(canceled)

### Claim 8

A non-transitory computer-readable medium,
bearing computer-readable instructions that upon execution on a computing device cause the computing device at least to:
extract a foot image from a target image displaying a foot;
generate foot posture information based on the foot image; and
superimpose, based on the foot posture information, a pattern of a clothing on the foot displayed in the target image.

### Claim 9 (depends on 6)

The method according to claim 6, the at least one memory further storing instructions that upon execution by the at least one processor cause the apparatus to:
input the target image into a pre-trained target detection network to obtain a first rectangular box;
expand, based on a center of the first rectangular box, the first rectangular box to obtain a second rectangular box; and
clip, from the target image, an image corresponding to the second rectangular box as the foot image.

### Claim 10 (depends on 6)

The method according to claim 6, the at least one memory further storing instructions that upon execution by the at least one processor cause the apparatus to:
input the foot image into a pre-trained foot pose estimation network to obtain foot pose information, wherein the foot pose information comprises: a quaternion of a position of the foot in the foot image relative to a position of an initial three-dimensional foot model, translation information of the position of the foot in the foot image relative to the position of the initial three-dimensional foot model, and left-right foot information in the foot image.

### Claim 11 (depends on 10)

The method according to claim 10, the at least one memory further storing instructions that upon execution by the at least one processor cause the apparatus to:
acquire a three-dimensional clothing model matching the initial three-dimensional foot model;
perform, based on the foot posture information, three-dimensional transformation on the three-dimensional clothing model by using a 3D engine, to superimpose an image of the three-dimensional clothing model on an image at the position of the foot displayed in the foot image, to obtain a superimposed foot image; and
obtain a rendered target image based on the superimposed foot image.

### Claim 12 (depends on 9)

The method according to claim 9, wherein the target detection network is one of the following:
a Fast R-CNN algorithm network, a Fast R-CNN algorithm network, a YOLO series algorithm network, and an SSD algorithm network.
