# US 11,113,511 B2 · Makeup evaluation system and operating method thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,113,511 B2
- Document ID: US11113511B2
- Status: Granted
- Kind: B2
- Application number: 16/482,511
- Filed: 2018-02-01
- Published: 2019-11-28
- Granted: 2021-09-07
- Earliest claimed: 2017-02-01
- Assignees: LG Household & Health Care Ltd.
- Applicants: LG Household & Health Care Ltd.
- Inventors: Sang E Kim; Do Hyuk Kwon; Do Sik Hwang; Tae Seong Kim; Doo Hyun Park; Ki Hun Bang; Tae Joon Eo; Yo Han Jun; Se Won Hwang
- Web page: https://patentgrep.com/patent/US11113511B2

## CPC

- G06V 10/82: using neural networks
- A45D 44/00: Other cosmetic or toiletry articles, e.g. for hairdressers' rooms
- A45D 44/005: {for selecting or displaying personal cosmetic colours or hairstyle}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06N 20/00: Machine learning
- G06Q 30/0643: {graphically representing goods, e.g. 3D product representation}
- G06Q 50/10: Services
- G06T 7/00: Image analysis
- G06T 11/26: Drawing of charts or graphs
- G06V 10/17: {using hand-held instruments}
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 10/56: relating to colour
- G06V 10/764: using classification, e.g. of video objects
- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- A45D 2044/007: {Devices for determining the condition of hair or skin or for selecting the appropriate cosmetic or hair treatment}
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10024: Color image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30201: Face
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A45D: HAIRDRESSING OR SHAVING EQUIPMENT (wigs, toupees or the like A41G3/00, A41G5/00; hairdressers' chairs A47C1/04; hair cutting appliances, razors B26B); EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A makeup evaluation system according to an embodiment of the present invention includes a mobile terminal for photographing a facial image and transmitting the photographed facial image to a makeup server, and the makeup server for storing makeup score data and, when receiving the facial image from the mobile terminal, detecting at least one face region in the facial image, calculating a makeup score for each of the detected face regions on the basis of the makeup score data, and transmitting the calculated makeup score to the mobile terminal.

## Claims

### Claim 1

A mobile terminal comprising:
a camera configured to photograph a facial image;
an input interface configured to receive an instruction to select a makeup theme corresponding to the facial image;
a wireless communication unit configured to transmit a makeup evaluation request signal corresponding to the facial image to a makeup server;
a control unit configured to receive a makeup score from the makeup server through the wireless communication unit; and
a display unit configured to display the makeup score according to the makeup evaluation request signal,
wherein the makeup score is acquired from makeup score data, is calculated for each of at least one face region detected in the facial image, and is calculated differently according to a shape of the detected face region and the makeup theme, and
wherein the makeup server, when calculating a first score which is the makeup score of the facial image and acquiring a second score which is a score of a makeup specialist for the facial image, includes a score data generation unit correcting the first score when a difference between the first score and the second score is equal to or greater than a reference value.

### Claim 2 (depends on 1)

The mobile terminal of claim 1, wherein the makeup server includes a makeup score learning unit for machine learning a makeup score calculation method through the facial image and the makeup score.

### Claim 3 (depends on 1)

The mobile terminal of claim 1, wherein the control unit acquires a highest makeup score among a plurality of makeup scores when the makeup score received from the makeup server is in plural, and the display unit displays the acquired highest makeup score.

### Claim 4 (depends on 3)

The mobile terminal of claim 3, wherein the display unit further displays a makeup theme corresponding to the highest makeup score.

### Claim 5 (depends on 1)

The mobile terminal of claim 1, wherein the wireless communication unit transmits the makeup evaluation request signal including the makeup theme to the makeup server and receives a makeup score corresponding to the makeup theme from the makeup server.

### Claim 6 (depends on 1)

The mobile terminal of claim 1, wherein the display unit displays at least one of the facial image, a makeup total score of the facial image, and a score for each face region of the facial image.

### Claim 7 (depends on 1)

The mobile terminal of claim 1, wherein the display unit displays the makeup score as a lowest point when the facial image is a no makeup image.

### Claim 8 (depends on 1)

The mobile terminal of claim 1, wherein the display unit, when the score of the face region is not a lowest point, displays a total score as the lowest point according to makeup balance.

### Claim 9 (depends on 1)

The mobile terminal of claim 1, wherein the display unit displays the same score in each of the facial images in which makeups are the same and photographing angles are different only.

### Claim 10 (depends on 1)

The mobile terminal of claim 1, wherein the makeup score data is data generated based on an input for a makeup evaluation of a makeup specialist.

### Claim 11 (depends on 10)

The mobile terminal of claim 10, wherein the makeup score data includes a makeup image by a first group of persons by a makeup theme, a makeup image by a second group of persons, and a makeup image by a third group of persons.

### Claim 12 (depends on 11)

The mobile terminal of claim 11, wherein the first group of persons includes general persons, the second group of persons includes beauty-related industry persons, and the third group of persons includes persons who are a makeup specialist.

### Claim 13

A mobile terminal comprising:
a camera configured to photograph a facial image;
an input interface configured to receive an instruction to select a makeup theme corresponding to the facial image;
a control unit configured to transmit a makeup evaluation request signal corresponding to the facial image; and
a display unit configured to display a makeup score according to the makeup evaluation request signal,
wherein the makeup score is acquired from makeup score data, is calculated for each of at least one face region detected in the facial image, and is calculated differently according to a shape of the detected face region and the makeup theme,
wherein the makeup score data is data generated based on an input for a makeup evaluation of a makeup specialist, and
wherein the makeup score data includes a makeup image by a first group of persons by a makeup theme, a makeup image by a second group of persons, and a makeup image by a third group of persons.