# US 2023/0274518 A1 · Augmented reality beauty product tutorials

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0274518 A1
- Document ID: US20230274518A1
- Status: Published application
- Kind: A1
- Application number: 18/142,759
- Filed: 2023-05-03
- Published: 2023-08-31
- Earliest claimed: 2020-03-31
- Assignees: Snap Inc.
- Applicants: Snap Inc.
- Inventors: Christine Barron; Virginia Drummond; Jean Luo; Alek Matthiessen; Celia Nicole Mourkogiannis; Jonathan Solichin; Olesia Voronova
- Web page: https://patentgrep.com/patent/US20230274518A1

## CPC

- G06T 11/60: Creating or editing images; Combining images with text
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/254: involving subtraction of images
- G06T 7/90: Determination of colour characteristics
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/20101: Interactive definition of point of interest, landmark or seed
- G06T 2207/30041: Eye; Retina; Ophthalmic
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Systems, methods, and computer readable media for augmented reality beauty product tutorials. Methods disclose determining from live images of an augmented reality (AR) tutorial effects, the effects indicating changes to the live images of a presenter of the AR tutorial from a beauty product being applied to a body part of the presenter. The methods further comprising determining from the live images motion, the motion indicating motion of the beauty product from the beauty product being applied to the body part of the presenter and storing the effects and the motion.

## Claims

### Claim 1

A system comprising:
at least one processor;
at least one computer screen coupled to the at least one processor;
at least one memory including instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising:
displaying, on a first portion of the at least one computer screen, a beauty product tutorial comprising pre-recorded images of a presenter applying a beauty product to a body part of the presenter, the beauty product tutorial further comprising effects to the presenter from applying the beauty product and indications of motion of the beauty product, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product;
processing live images of a user to add the effects and the indications of motion of the beauty product; and
displaying on a second portion of the at least one computer screen the processed live images of the user, wherein images of the beauty product tutorial and the processed live images are displayed so that the effects are displayed simultaneously on the second portion of the at least one computer screen with corresponding effects of the beauty product tutorial on the first portion of the at least one computer screen.

### Claim 2 (depends on 1)

The system of claim 1, wherein the operations further comprise:
determining the effects to the presenter from applying the beauty product, the effects indicating changes to the images of the presenter from the beauty product being applied to the body part of the presenter; and
determining, based on the effects, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product.

### Claim 3 (depends on 2)

The system of claim 2, wherein the indications of motion are further determined based on a predetermined color of the beauty product.

### Claim 4 (depends on 2)

The system of claim 2, wherein the effects are further determined based on a predetermined color of the beauty product.

### Claim 5 (depends on 2)

The system of claim 2, wherein the effects are determined based on comparing successive images of the presenter.

### Claim 6 (depends on 2)

The system of claim 2, wherein the effects are determined using a neural network.

### Claim 7 (depends on 2)

The system of claim 2, wherein the effects and the determined motion are stored to indicate changes from a first image of the live images to a second image of the live images.

### Claim 8 (depends on 1)

The system of claim 1, wherein the operations further comprise:
determining from the beauty product tutorial the body part of the presenter; and
adjusting the processed live images of the user to zoom in on the body part of the user.

### Claim 9 (depends on 8)

The system of claim 8, wherein the operations further comprise:
determining the body part based on predetermined images of body parts.

### Claim 10 (depends on 1)

The system of claim 1, wherein the processing the live images of the user to add the effects further comprises:
modifying the effects in accordance with a position of a body part of the user.

### Claim 11 (depends on 1)

The system of claim 1, wherein the beauty product tutorial comprises a number of steps, and wherein the operations further comprise:
proceeding to a next step based on input received from the user.

### Claim 12

A method comprising:
displaying, on a first portion of a screen, a beauty product tutorial comprising pre-recorded images of a presenter applying a beauty product to a body part of the presenter, the beauty product tutorial further comprising effects to the presenter from applying the beauty product and indications of motion of the beauty product, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product;
processing live images of a user to add the effects and the indications of motion of the beauty product; and
displaying on a second portion of the screen the processed live images of the user, wherein images of the beauty product tutorial and the processed live images are displayed so that the effects are displayed simultaneously on the second portion of the screen with corresponding effects of the beauty product tutorial on the first portion of the screen.

### Claim 13 (depends on 12)

The method of claim 12, wherein the method further comprises:
determining the effects to the presenter from applying the beauty product, the effects indicating changes to the images of the presenter from the beauty product being applied to the body part of the presenter; and
determining, based on the effects, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product.

### Claim 14 (depends on 13)

The method of claim 13, wherein the indications of motion are further determined based on a predetermined color of the beauty product.

### Claim 15 (depends on 13)

The method of claim 13, wherein the effects are further determined based on a predetermined color of the beauty product.

### Claim 16 (depends on 13)

The method of claim 13, wherein the effects are determined based on comparing successive images of the presenter.

### Claim 17

A non-transitory computer-readable medium comprising instructions, which when executed by one or more processors, cause the one or more processors to perform operations comprising:
displaying, on a first portion of a screen, a beauty product tutorial comprising pre-recorded images of a presenter applying a beauty product to a body part of the presenter, the beauty product tutorial further comprising effects to the presenter from applying the beauty product and indications of motion of the beauty product, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product;
processing live images of a user to add the effects and the indications of motion of the beauty product; and
displaying on a second portion of the screen the processed live images of the user, wherein images of the beauty product tutorial and the processed live images are displayed so that the effects are displayed simultaneously on the second portion of the screen with corresponding effects of the beauty product tutorial on the first portion of the screen.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein the operations further comprise:
determining the effects to the presenter from applying the beauty product, the effects indicating changes to the images of the presenter from the beauty product being applied to the body part of the presenter; and
determining, based on the effects, the indications of motion of the beauty product indicating the motion used by the presenter in applying the beauty product.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the indications of motion are further determined based on a predetermined color of the beauty product.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein the effects are further determined based on a predetermined color of the beauty product.
