Back to results
US 11,900,506 B2Granted
Controlling interactive fashion based on facial expressions
AssigneeSnap Inc.
InventorsItamar Berger, Gal Dudovitch, Gal Sasson, Ma'ayan Shuvi, Matan Zohar
Life of this patent
Estimating the term…
Abstract
Methods and systems are disclosed for performing operations comprising: receiving an image that includes a depiction of a person wearing a fashion item; generating a segmentation of the fashion item worn by the person depicted in the image; identifying a facial expression of the user depicted in the image; and in response to identifying the facial expression, applying one or more augmented reality elements to the fashion item worn by the person based on the segmentation of the fashion item worn by the person.
Claims20
A method comprising:
- receiving, by one or more processors of a client device, an image that includes a depiction of a person wearing a fashion item;
- generating, by the one or more processors, a segmentation of the fashion item worn by the person depicted in the image;
- detecting a phrase physically written on the fashion item worn by the person depicted in the image;
- processing a facial expression of the person depicted in the image using a machine learning model to estimate an emotion or mood of the person depicted in the image;
- identifying a word that corresponds to the emotion or mood of the person depicted in the image that has been estimated by processing the facial expression using the machine learning model;
- generating an augmented reality element comprising the identified word that corresponds to the emotion or mood of the person that has been estimated by processing the facial expression using the machine learning model;
- and
- in response to determining that the image includes the depiction of the person wearing the fashion item, replacing a portion of the phrase physically written on the fashion item with the generated augmented reality element comprising the identified word.
5 more elements
A system comprising:
- a processor of a client device;
- and
- a memory component having instructions stored thereon, when executed by the processor, causes the processor to perform operations comprising:
- receiving an image that includes a depiction of a person wearing a fashion item;
- generating a segmentation of the fashion item worn by the person depicted in the image;
- detecting a phrase physically written on the fashion item worn by the person depicted in the image;
- processing a facial expression of the person depicted in the image using a machine learning model to estimate an emotion or mood of the person depicted in the image;
- identifying a word that corresponds to the emotion or mood of the person depicted in the image that has been estimated by processing the facial expression using the machine learning model;
- generating an augmented reality element comprising the identified word that corresponds to the emotion or mood of the person that has been estimated by processing the facial expression using the machine learning model;
- and
- in response to determining that the image includes the depiction of the person wearing the fashion item, replacing a portion of the phrase physically written on the fashion item with the generated augmented reality element comprising the identified word.
8 more elements
A non-transitory computer-readable storage medium having stored thereon instructions that, when executed by a processor of a client device, cause the processor to perform operations comprising:
- receiving an image that includes a depiction of a person wearing a fashion item;
- generating a segmentation of the fashion item worn by the person depicted in the image;
- detecting a phrase physically written on the fashion item worn by the person depicted in the image;
- processing a facial expression of the person depicted in the image using a machine learning model to estimate an emotion or mood of the person depicted in the image;
- identifying a word that corresponds to the emotion or mood of the person depicted in the image that has been estimated by processing the facial expression using the machine learning model;
- generating an augmented reality element comprising the identified word that corresponds to the emotion or mood of the person that has been estimated by processing the facial expression using the machine learning model;
- and
- in response to determining that the image includes the depiction of the person wearing the fashion item, replacing a portion of the phrase physically written on the fashion item with the generated augmented reality element comprising the identified word.
5 more elements
Dependencies are inferred from the claim text.