# US 2022/0284827 A1 · Systems and methods for generating individualized cosmetic programs utilizing intelligent feedback

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284827 A1
- Document ID: US20220284827A1
- Status: Published application
- Kind: A1
- Application number: 17/190,066
- Filed: 2021-03-02
- Published: 2022-09-08
- Earliest claimed: 2021-03-02
- Inventors: Regina M. GARCIA; Eric R. GARCIA
- Web page: https://patentgrep.com/patent/US20220284827A1

## CPC

- A45D 44/005: {for selecting or displaying personal cosmetic colours or hairstyle}
- A45D 42/08: Shaving mirrors
- G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/68: of symmetry
- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- G06T 11/23: using straight lines or curves
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G09B 5/02: with visual presentation of the material to be studied, e.g. using film strip
- G09B 5/065: {Combinations of audio and video presentations, e.g. videotapes, videodiscs, television systems}
- G09B 19/00: Teaching not covered by other main groups of this subclass (teaching or practice apparatus for gun-aiming or gun-laying F41G3/26)
- G09B 19/003: {Repetitive work cycles; Sequence of movements}
- G09B 19/0076: {Body hygiene; Dressing; Knot tying}
- A45D 42/00: Hand, pocket, or shaving mirrors
- A45D 2044/007: {Devices for determining the condition of hair or skin or for selecting the appropriate cosmetic or hair treatment}
- G06F 2203/04803: Split screen, i.e. subdividing the display area or the window area into separate subareas
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/30088: Skin; Dermal
- G06T 2207/30201: Face
- A45D: HAIRDRESSING OR SHAVING EQUIPMENT (wigs, toupees or the like A41G3/00, A41G5/00; hairdressers' chairs A47C1/04; hair cutting appliances, razors B26B); EQUIPMENT FOR COSMETICS OR COSMETIC TREATMENTS, e.g. FOR MANICURING OR PEDICURING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B)

## Abstract

A smart mirror includes a mirror having a reflective surface and a transparent surface; an image capture device configured to capture images of the user; a display disposed proximate to the transparent surface of the mirror, the display configured to augment a reflection of the user; and a processor configured to: analyze images of the user to identify a plurality of reference points corresponding to facial features of the user; assess a symmetry of the facial features of the user using the plurality of reference points; generate an individualized cosmetic treatment program based on the symmetry of the facial features; and cause the display to augment a reflection of the user with a first cosmetic instruction, the first cosmetic instruction configured to aid the user in applying a cosmetic material onto a first area of a face of the user according to the individualized cosmetic treatment program.

## Claims

### Claim 1

A method for generating an individualized cosmetic program, the method comprising:
capturing an image of a user, the image including facial features of the user;
analyzing the image to identify a plurality of reference points corresponding to the facial features of the user;
assessing a symmetry of the facial features of the user using the plurality of reference points;
generating an individualized cosmetic treatment program based on the symmetry of the facial features; and
displaying a first cosmetic instruction based on the individualized cosmetic treatment program, the first cosmetic instruction configured to aid the user in applying a cosmetic material onto a first area of a face of the user.

### Claim 2 (depends on 1)

The method of claim 1, further comprising providing a first intervention to alter the application of the cosmetic material in response to a detected misapplication of the cosmetic material.

### Claim 3 (depends on 1)

The method of claim 1, further comprising parsing the individualized cosmetic treatment program into a plurality of segments, wherein the first cosmetic instruction corresponds to a first segment of the plurality of segments.

### Claim 4 (depends on 3)

The method of claim 3, further comprising modifying the display of the first cosmetic instruction as the user progresses through the first segment of the plurality of segments.

### Claim 5 (depends on 3)

The method of claim 3, further comprising:
displaying a second cosmetic instruction corresponding to a second segment of the plurality of segments, the second cosmetic instruction configured to aid the user in applying the cosmetic material onto a second area of the face of the user;
monitoring application of the cosmetic material based on the second cosmetic instruction to detect a misapplication of the cosmetic material;
providing a second intervention to alter the application of the cosmetic material in response to a detected misapplication of the cosmetic material; and
modifying the display of the second cosmetic instruction as the user progresses through the second segment of the plurality of segments.

### Claim 6 (depends on 1)

The method of claim 1, further comprising identifying a skin condition of the user, wherein the individualized cosmetic treatment program is further generated based on the identified skin condition.

### Claim 7 (depends on 1)

The method of claim 1, further comprising modifying a rendering the first cosmetic instruction based on a change in an orientation of the face of the user.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first cosmetic instruction comprises at least one of an outline delineating an area for application of the cosmetic material and a color indicating a shade for application of the cosmetic material.

### Claim 9 (depends on 8)

The method of claim 8, further comprising displaying a tutorial video corresponding to the first cosmetic instruction.

### Claim 10 (depends on 1)

The method of claim 1, wherein the displaying comprises augmenting a reflection of the user with the first cosmetic instruction.

### Claim 11

A smart mirror for generating an individualized cosmetic program, the smart mirror comprising:
a mirror having a reflective surface on one side and a transparent surface on an opposite side;
an image capture device configured to capture images of a user;
a display disposed proximate to the transparent surface of the mirror, the display configured to augment a reflection of the user;
a processor configured to:
analyze images of the user to identify a plurality of reference points corresponding to facial features of the user;
assess a symmetry of the facial features of the user using the plurality of reference points;
generate an individualized cosmetic treatment program based on the symmetry of the facial features; and
cause the display to augment a reflection of the user with a first cosmetic instruction, the first cosmetic instruction configured to aid the user in applying a cosmetic material onto a first area of a face of the user according to the individualized cosmetic treatment program.

### Claim 12 (depends on 11)

The smart mirror of claim 11, further comprising a touch interface for receiving a selection of a cosmetic routine, skin care routine, or health screening routine.

### Claim 13 (depends on 11)

The smart mirror of claim 11, further comprising a light for illuminating the user, wherein the processor is further configured to adjust a color temperature and intensity of emitted light based on a color temperature or intensity of ambient light.

### Claim 14 (depends on 11)

The smart mirror of claim 11, wherein the processor is further configured to provide a first intervention to alter the application of the cosmetic material in response to a detected misapplication of the cosmetic material.

### Claim 15 (depends on 11)

The smart mirror of claim 11, wherein the processor is further configured to parse the individualized cosmetic treatment program into a plurality of segments, wherein the first cosmetic instruction corresponds to a first segment of the plurality of segments.

### Claim 16 (depends on 15)

The smart mirror of claim 15, wherein the processor is further configured to modify the display of the first cosmetic instruction as the user progresses through the first segment of the plurality of segments.

### Claim 17 (depends on 11)

The smart mirror of claim 11, wherein the processor is further configured to identify a skin condition of the user, wherein the individualized cosmetic treatment program is further generated based on the identified skin condition.

### Claim 18 (depends on 11)

The smart mirror of claim 11, wherein the first cosmetic instruction comprises at least one of an outline delineating an area for application of the cosmetic material and a color indicating a shade for application of the cosmetic material.

### Claim 19 (depends on 18)

The smart mirror of claim 18, wherein the processor is further configured to cause the display to play a tutorial video corresponding to the first cosmetic instruction.

### Claim 20

A tangible, non-transitory, computer-readable media having instructions encoded thereon, the instructions, when executed by a processor, is operable to:
capture an image of a user, the image including facial features of the user;
analyze the image to identify a plurality of reference points corresponding to the facial features of the user;
assess a symmetry of the facial features of the user using the plurality of reference points;
generate an individualized cosmetic treatment program based on the symmetry of the facial features; and
display a first cosmetic instruction based on the individualized cosmetic treatment program, the first cosmetic instruction configured to aid the user in applying a cosmetic material onto a first area of a face of the user.
