# US 2019/0244274 A1 · Systems and methods for recommending products based on facial analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0244274 A1
- Document ID: US20190244274A1
- Status: Published application
- Kind: A1
- Application number: 15/986,941
- Filed: 2018-05-23
- Published: 2019-08-08
- Earliest claimed: 2018-02-06
- Assignees: Perfect Corp.
- Applicants: Perfect Corp.
- Inventors: Tzu-Chieh Chang; Yi-Wei Lin; Pei-Wen Huang; Ping-Xing Chen
- Web page: https://patentgrep.com/patent/US20190244274A1

## CPC

- G06Q 30/0631: {Recommending goods or services}
- G06Q 30/0641: {utilising user interfaces specially adapted for shopping}
- G06T 7/50: Depth or shape recovery
- G06T 7/62: of area, perimeter, diameter or volume
- G06T 7/90: Determination of colour characteristics
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06T 2207/30201: Face
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

In a computing device for recommending products based on facial analysis, use of a front-facing camera of the computing device is monitored. In response to detecting use of the front-facing camera to capture a self-portrait image, the computing device analyzes facial features of a facial region of an individual depicted in the image captured by the front-facing camera. The computing device further accesses corresponding measurement templates for the facial features and apples at least one of the measurement templates to corresponding facial features. The computing device retrieves product identifiers for the facial features corresponding to the at least one applied measurement template. The computing device generates at least one product recommendation based on the retrieved product identifiers and displays the at least one product recommendation on a user interface.

## Claims

### Claim 1

A method implemented in a computing device for recommending products based on facial analysis, comprising:
monitoring use of a front-facing camera of the computing device;
responsive to detecting use of the front-facing camera to capture a self-portrait image, performing the steps of:
analyzing facial features of a facial region of an individual depicted in the image captured by the front-facing camera;
accessing corresponding measurement templates for the facial features;
applying at least one of the measurement templates to corresponding facial features;
retrieving product identifiers for the facial features corresponding to the at least one applied measurement template;
generating at least one product recommendation based on the retrieved product identifiers; and
displaying the at least one product recommendation on a user interface.

### Claim 2 (depends on 1)

The method of claim 1, wherein each of the measurement templates specifies one or more target attributes to analyze for a corresponding facial feature.

### Claim 3 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises eyes in the facial region, and wherein the one or more target attributes for the measurement template comprise distance measurements involving pairs of points located around a boundary of each eye.

### Claim 4 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises eyes in the facial region, and wherein the one or more target attributes for the measurement template comprise curvature measurements at a point on a boundary of each eye for determining whether the eyes are approximately almond shaped or round.

### Claim 5 (depends on 3)

The method of claim 3, wherein the one or more target attributes for the measurement template comprise a ratio between a width of each eye and a height of each eye.

### Claim 6 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises skin in the facial region, and wherein the one or more target attributes for the measurement template comprise skin tone.

### Claim 7 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises the entire facial region, and wherein the one or more target attributes for the measurement template comprise a face shape.

### Claim 8 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises lips in the facial region, and wherein the one or more target attributes for the measurement template comprise a lip contour.

### Claim 9 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises lips in the facial region, and wherein the one or more target attributes for the measurement template comprise a lip color.

### Claim 10 (depends on 2)

The method of claim 2, wherein the corresponding facial feature comprises lips in the facial region, and wherein the one or more target attributes for the measurement template comprise a thickness of an upper lip portion and a thickness of a lower lip portion.

### Claim 11 (depends on 1)

The method of claim 1, wherein displaying the at least one product recommendation on the user interface is performed based on weight values assigned to the facial features.

### Claim 12 (depends on 11)

The method of claim 11, wherein the weight values comprise one of:
predefined values; or values specified by a user.

### Claim 13 (depends on 1)

The method of claim 1, wherein the facial features comprise facial features selected by a user of the computing device.

### Claim 14

A system, comprising:
a front-facing camera;
a memory storing instructions; and
a processor coupled to the memory and configured by the instructions to at least:
monitor use of the front-facing camera;
responsive to detecting use of the front-facing camera to capture a self-portrait image, perform the steps of:
analyzing facial features of a facial region of an individual depicted in the image captured by the front-facing camera;
accessing corresponding measurement templates for the facial features;
applying at least one of the measurement templates to corresponding facial features;
retrieving product identifiers for the facial features corresponding to the at least one applied measurement template;
generating at least one product recommendation based on the retrieved product identifiers; and
displaying the at least one product recommendation on a user interface.

### Claim 15 (depends on 14)

The system of claim 14, wherein each of the measurement templates specifies one or more target attributes to analyze for a corresponding facial feature.

### Claim 16 (depends on 15)

The system of claim 15, wherein the corresponding facial feature comprises eyes in the facial region, and wherein the one or more target attributes for the measurement template comprise distance measurements involving pairs of points located around a boundary of each eye.

### Claim 17 (depends on 15)

The system of claim 15, wherein the corresponding facial feature comprises skin in the facial region, and wherein the one or more target attributes for the measurement template comprise skin tone.

### Claim 18 (depends on 15)

The system of claim 15, wherein the corresponding facial feature comprises the entire facial region, and wherein the one or more target attributes for the measurement template comprise a face shape.

### Claim 19 (depends on 15)

The system of claim 15, wherein the corresponding facial feature comprises lips in the facial region, and wherein the one or more target attributes for the measurement template comprise a thickness of an upper lip portion and a thickness of a lower lip portion.

### Claim 20

A non-transitory computer-readable storage medium storing instructions to be implemented by a computing device having a processor, wherein the instructions, when executed by the processor, cause the computing device to at least:
monitor use of a front-facing camera of the computing device;
responsive to detecting use of the front-facing camera to capture a self-portrait image, perform the steps of:
analyzing facial features of a facial region of an individual depicted in the image captured by the front-facing camera;
accessing corresponding measurement templates for the facial features;
applying at least one of the measurement templates to corresponding facial features;
retrieving product identifiers for the facial features corresponding to the at least one applied measurement template;
generating at least one product recommendation based on the retrieved product identifiers; and
displaying the at least one product recommendation on a user interface.

### Claim 21 (depends on 20)

The non-transitory computer-readable storage medium of claim 20, wherein each of the measurement templates specifies one or more target attributes to analyze for a corresponding facial feature.
