# US 2019/0244015 A1 · Systems and methods for generating and analyzing user behavior metrics during makeup consultation sessions

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244015 A1
- Document ID: US20190244015A1
- Status: Published application
- Kind: A1
- Application number: 16/008,217
- Filed: 2018-06-14
- Published: 2019-08-08
- Earliest claimed: 2018-02-06
- Assignees: Perfect Corp.
- Applicants: Perfect Corp.
- Inventors: Wan-Chuan Lee
- Web page: https://patentgrep.com/patent/US20190244015A1

## CPC

- G06Q 30/02: Marketing; Price estimation or determination; Fundraising
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 20/46: {Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- H04L 12/1813: {for computer conferences, e.g. chat rooms (instant messaging H04L51/04; protocols for multimedia communication H04L65/1101; arrangements for multi-party communication H04L65/403; telephonic conference arrangements H04M3/56; television conference systems H04N7/15)}
- H04N 7/141: {between two video terminals, e.g. videophone (telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085)}
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A server device detects initiation of a video conferencing session between a consultation device utilized by a makeup professional and a client device utilized by a user receiving a makeup consultation from the makeup professional. The server device extracts data from the client device during the video conferencing session, the data characterizing behavior of the user performed on the client device with respect to suggested cosmetic effects transmitted by the makeup professional via the consultation device to the client device. The server device applies weight values to the extracted data and generates one or more hesitation metrics based on the weight values and causes the one or more hesitation metrics to be displayed in a user interface on the consultation device.

## Claims

### Claim 1

A method implemented in a server device, comprising:
detecting initiation of a video conferencing session between a consultation device utilized by a makeup professional and a client device utilized by a user receiving a makeup consultation from the makeup professional;
extracting data from the client device during the video conferencing session, the data characterizing behavior of the user performed on the client device with respect to suggested cosmetic effects transmitted by the makeup professional via the consultation device to the client device;
applying weight values to the extracted data;
generating one or more hesitation metrics based on the weight values; and
causing the one or more hesitation metrics to be displayed in a user interface on the consultation device.

### Claim 2 (depends on 1)

The method of claim 1, wherein extracting the data from the client device comprises extracting data relating to a predetermined grouping of target events corresponding to user behavior on the client device.

### Claim 3 (depends on 2)

The method of claim 2, wherein applying the weight values to the extracted data comprises applying predetermined weight values for each event in the grouping of target events.

### Claim 4 (depends on 3)

The method of claim 3, wherein the grouping of target events comprises at least one of:
selection by the user on the client device of a type of cosmetic effect;
selection by the user on the client device of a variation in attribute of the selected type of cosmetic effect; and
removal by the user on the client device of a type of cosmetic effect.

### Claim 5 (depends on 4)

The method of claim 4, wherein each type of cosmetic effect corresponds to cosmetic effects for different facial features.

### Claim 6 (depends on 4)

The method of claim 4, wherein the variation in the attribute of the selected type of cosmetic effect comprises a color variation of the type of cosmetic effect.

### Claim 7 (depends on 4)

The method of claim 4, wherein the variation in the attribute of the selected type of cosmetic effect comprises enhancement or reduction of a cosmetic effect.

### Claim 8 (depends on 1)

The method of claim 1, wherein causing the one or more hesitation metrics to be displayed in the user interface on the consultation device comprises:
sorting the one or more hesitation metrics; and
causing the sorted one or more hesitation metrics to be displayed in the user interface on the consultation device.

### Claim 9

A system, comprising:
a memory storing instructions;
a processor coupled to the memory and configured by the instructions to at least:
detect initiation of a video conferencing session between a consultation device utilized by a makeup professional and a client device utilized by a user receiving a makeup consultation from the makeup professional;
extract data from the client device during the video conferencing session, the data characterizing behavior of the user performed on the client device with respect to suggested cosmetic effects transmitted by the makeup professional via the consultation device to the client device;
apply weight values to the extracted data and generate one or more hesitation metrics based on the weight values; and
cause the one or more hesitation metrics to be displayed in a user interface on the consultation device.

### Claim 10 (depends on 9)

The system of claim 9, wherein the processor extracts the data from the client device by extracting data relating to a predetermined grouping of target events corresponding to user behavior on the client device.

### Claim 11 (depends on 10)

The system of claim 10, wherein the processor applies the weight values to the extracted data by applying predetermined weight values for each event in the grouping of target events.

### Claim 12 (depends on 11)

The system of claim 11, wherein the grouping of target events comprises at least one of:
selection by the user on the client device of a type of cosmetic effect;
selection by the user on the client device of a variation in attribute of the selected type of cosmetic effect; and
removal by the user on the client device of a type of cosmetic effect.

### Claim 13 (depends on 12)

The system of claim 12, wherein each type of cosmetic effect corresponds to cosmetic effects for different facial features.

### Claim 14 (depends on 12)

The system of claim 12, wherein the variation in the attribute of the selected type of cosmetic effect comprises a color variation of the type of cosmetic effect.

### Claim 15 (depends on 9)

The system of claim 9, wherein the processor causes the one or more hesitation metrics to be displayed in the user interface on the consultation device by:
sorting the one or more hesitation metrics; and
causing the sorted one or more hesitation metrics to be displayed in the user interface on the consultation device.

### Claim 16

A non-transitory computer-readable storage medium storing instructions to be implemented by a computing device having a processor, wherein the instructions, when executed by the processor, cause the computing device to at least:
detect initiation of a video conferencing session between a consultation device utilized by a makeup professional and a client device utilized by a user receiving a makeup consultation from the makeup professional;
extract data from the client device during the video conferencing session, the data characterizing behavior of the user performed on the client device with respect to suggested cosmetic effects transmitted by the makeup professional via the consultation device to the client device;
apply weight values to the extracted data and generate one or more hesitation metrics based on the weight values; and
cause the one or more hesitation metrics to be displayed in a user interface on the consultation device.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium of claim 16, wherein the processor extracts the data from the client device by extracting data relating to a predetermined grouping of target events corresponding to user behavior on the client device.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium of claim 17, wherein the processor applies the weight values to the extracted data by applying predetermined weight values for each event in the grouping of target events.

### Claim 19 (depends on 18)

The non-transitory computer-readable storage medium of claim 18, wherein the grouping of target events comprises at least one of:
selection by the user on the client device of a type of cosmetic effect;
selection by the user on the client device of a variation in attribute of the selected type of cosmetic effect; and
removal by the user on the client device of a type of cosmetic effect.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein each type of cosmetic effect corresponds to cosmetic effects for different facial features.

### Claim 21 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, wherein the variation in the attribute of the selected type of cosmetic effect comprises a color variation of the type of cosmetic effect.
