# US 2017/0255478 A1 · Systems and methods for simulated application of cosmetic effects

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0255478 A1
- Document ID: US20170255478A1
- Status: Published application
- Kind: A1
- Application number: 15/446,301
- Filed: 2017-03-01
- Published: 2017-09-07
- Earliest claimed: 2016-03-03
- Assignees: Perfect Corp.
- Applicants: Perfect Corp.
- Inventors: Chen-Wei Chou; Ching-Hsuan Ma
- Web page: https://patentgrep.com/patent/US20170255478A1

## CPC

- G06F 9/453: {Help systems}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/012: {Head tracking input arrangements}
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 30/20: Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30)
- G06Q 30/06: Buying, selling or leasing transactions
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

In a computing device for simulating the application of cosmetic effects, a user interface with a plurality of cosmetic templates is displayed to a user, where each cosmetic template comprises a predefined sequence for applying cosmetic effects. A selection of one of the cosmetic templates is obtained from the user, and a digital representation of the user is captured. The computing device tracks facial features of the user in the digital representation of the user and generates a virtual mirror for the user to view. The virtual mirror displays progressive application of each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features in the digital representation. A digital representation of the simulated application of all of the sequence of cosmetic effects is then stored.

## Claims

### Claim 1

A method implemented in a computing device for simulating application of cosmetic effects, comprising:
displaying, by the computing device, a user interface with a plurality of cosmetic templates to a user, each cosmetic template comprising a predefined sequence for applying cosmetic effects;
obtaining, by the computing device, a selection of one of the cosmetic templates from the user;
capturing, by the computing device, a digital representation of the user;
tracking, by the computing device, facial features of the user in the digital representation of the user;
generating, by the computing device, a virtual mirror for the user to view, the virtual mirror displaying progressive application of each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features in the digital representation; and
storing, by the computing device, a digital representation of the simulated application of all of the sequence of cosmetic effects.

### Claim 2 (depends on 1)

The method of claim 1, wherein the virtual mirror further displays at least one of: retailer information relating to a cosmetic product associated with each cosmetic effect in the sequence, and color information relating to the cosmetic product.

### Claim 3 (depends on 1)

The method of claim 1, wherein displaying, by the computing device, the user interface with the plurality of cosmetic templates to the user further comprises displaying a preview window depicting a post-makeup result with the selected cosmetic template applied to the user's face.

### Claim 4 (depends on 1)

The method of claim 1, wherein the virtual mirror further displays instructions guiding the user how to apply each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features.

### Claim 5 (depends on 4)

The method of claim 4 wherein the instructions comprise at least one of: text displayed on portions of the user's face, annotation, animation, step-by-step directions, information relating to color intensity of one of more of the cosmetic effects within the sequence of cosmetic effects, and text describing target facial regions for applying each of the sequence of cosmetic effects.

### Claim 6 (depends on 4)

The method of claim 4, wherein the virtual mirror comprises a split screen display, wherein a first half of the split screen corresponds to simulated application of each cosmetic effect to one of a right side or a left side of the user's face depicted in the digital representation, and wherein the second half of the split screen corresponds to actual application by the user of each cosmetic effect to the other of the right side or the left side of the user's face.

### Claim 7 (depends on 6)

The method of claim 6, wherein the two halves of the split screen display are shown side by side to provide the user with a comparison between the simulated application of each cosmetic effect and the actual application by the user of each cosmetic effect.

### Claim 8 (depends on 4)

The method of claim 4, wherein the virtual mirror comprises a dual display configuration, wherein a first display corresponds to simulated application of each cosmetic effect to the user's face depicted in the digital representation, and wherein the second display corresponds to actual application by the user of each cosmetic effect to the user's face.

### Claim 9 (depends on 8)

The method of claim 8, wherein the first display shows the simulated application of each cosmetic effect is on a digital representation captured prior to application of any makeup effect, and wherein the second display shows a live video of the user's face during the actual application by the user of each cosmetic effect.

### Claim 10 (depends on 8)

The method of claim 8, wherein only one of the first display or the second display is shown at a time based on manipulation of a toggle control on the user interface.

### Claim 11 (depends on 4)

The method of claim 4, wherein the virtual mirror comprises a split screen display configuration, wherein a first display displays the user's face without application of any cosmetic effect, and wherein the second display displays the user's face during simulated application of each cosmetic effect.

### Claim 12 (depends on 1)

The method of claim 1, wherein capturing the digital representation of the user comprises configuring a front facing camera of computing device to capture live video of the user.

### Claim 13

A system, comprising:
a memory storing instructions; and
a processor coupled to the memory and configured by the instructions to at least:
display a user interface with a plurality of cosmetic templates to a user, each cosmetic template comprising a predefined sequence for applying cosmetic effects;
obtain a selection of one of the cosmetic templates from the user;
capture a digital representation of the user;
track facial features of the user in the digital representation of the user;
generate a virtual mirror for the user to view, the virtual mirror displaying progressive application of each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features in the digital representation; and
store a digital representation of the simulated application of all of the sequence of cosmetic effects.

### Claim 14 (depends on 13)

The system of claim 13, wherein the virtual mirror further displays at least one of: retailer information relating to a cosmetic product associated with each cosmetic effect in the sequence, and color information relating to the cosmetic product.

### Claim 15 (depends on 13)

The system of claim 13, wherein displaying the user interface with the plurality of cosmetic templates to the user further comprises displaying a preview window depicting a post-makeup result with the selected cosmetic template applied to the user's face.

### Claim 16 (depends on 13)

The system of claim 13, wherein the virtual mirror further displays instructions guiding the user how to apply each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features.

### Claim 17 (depends on 16)

The system of claim 16, wherein the virtual mirror comprises a split screen display, wherein a first half of the split screen corresponds to simulated application of each cosmetic effect to one of a right side or a left side of the user's face depicted in the digital representation, and wherein the second half of the split screen corresponds to actual application by the user of each cosmetic effect to the other of the right side or the left side of the user's face.

### Claim 18 (depends on 17)

The system of claim 17, wherein the two halves of the split screen display are shown side by side to provide the user with a comparison between the simulated application of each cosmetic effect and the actual application by the user of each cosmetic effect.

### Claim 19 (depends on 16)

The system of claim 16, wherein the virtual mirror comprises a dual display configuration, wherein a first display corresponds to simulated application of each cosmetic effect to the user's face depicted in the digital representation, and wherein the second display corresponds to actual application by the user of each cosmetic effect to the user's face.

### Claim 20 (depends on 19)

The system of claim 19, wherein the first display shows the simulated application of each cosmetic effect is on a digital representation captured prior to application of any makeup effect, and the second display shows a live video of the user's face during the actual application by the user of each cosmetic effect.

### Claim 21

A non-transitory computer-readable storage medium storing instructions to be implemented by a computing device having a processor, wherein the instructions, when executed by the processor, cause the computing device to at least:
display a user interface with a plurality of cosmetic templates to a user, each cosmetic template comprising a predefined sequence for applying cosmetic effects;
obtain a selection of one of the cosmetic templates from the user;
capture a digital representation of the user;
track facial features of the user in the digital representation of the user;
generate a virtual mirror for the user to view, the virtual mirror displaying progressive application of each of the sequence of cosmetic effects of the selected cosmetic template to corresponding facial features in the digital representation; and
store a digital representation of the simulated application of all of the sequence of cosmetic effects.

### Claim 22 (depends on 21)

The non-transitory computer-readable storage medium of claim 21, wherein the virtual mirror comprises a split screen display, wherein a first half of the split screen corresponds to simulated application of each cosmetic effect to one of a right side or a left side of the user's face depicted in the digital representation, and wherein the second half of the split screen corresponds to actual application by the user of each cosmetic effect to the other of the right side or the left side of the user's face.

### Claim 23 (depends on 21)

The non-transitory computer-readable storage medium of claim 21, wherein the virtual mirror comprises a dual display configuration, wherein a first display corresponds to simulated application of each cosmetic effect to the user's face depicted in the digital representation, and wherein the second display corresponds to actual application by the user of each cosmetic effect to the user's face.
