Back to results
US 12,469,196 B2Granted
Augmented expression system
AssigneeSnap Inc.
InventorsChen Cao, Yang Gao, Zehao Xue
Life of this patent
Estimating the term…
Abstract
Embodiments described herein relate to an augmented expression system to generate and cause display of a specially configured interface to present an augmented reality perspective. The augmented expression system receives image and video data of a user and tracks facial landmarks of the user based on the image and video data, in real-time to generate and present a 3-dimensional (3D) bitmoji of the user.
Claims8
A method comprising:
- causing display of image data at a client device, the image data comprising a set of facial landmarks;
- detecting the set of facial landmarks within the image data;
- and
- responsive to the detecting the set of facial landmarks within the image data, performing operations comprising:
- identifying a user profile based on attributes of the set of facial landmarks from within the image data;
- accessing user profile data associated with the user profile, the user profile data including an expression definition that maps facial tracking inputs that comprise a landmark identifier, an input value indicating movement relative to a static position, and a movement direction to a predefined expression that includes a corresponding graphical element;
- determining an expression of a user based on a combination of facial tracking inputs that include simultaneous movements of multiple facial landmarks relative to one another, the combination of facial tracking inputs corresponding to the predefined expression;
- generating a graphical depiction of the user based on the graphical element that corresponds with the movements and the positions of the facial landmarks relative to one another;
- and
- overlaying the graphical depiction of the user at a position upon the image data at the client device.
7 more elements
A system comprising:
- a memory;
- and
- at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising:
- causing display of image data at a client device, the image data comprising a set of facial landmarks;
- detecting the set of facial landmarks within the image data;
- and
- responsive to the detecting the set of facial landmarks within the image data, performing operations comprising:
- identifying a user profile based on attributes of the set of facial landmarks from within the image data;
- accessing user profile data associated with the user profile, the user profile data including an expression definition that maps facial tracking inputs that comprise a landmark identifier, an input value indicating movement relative to a static position, and a movement direction to a predefined expression that includes a corresponding graphical element;
- determining an expression of a user based on a combination of facial tracking inputs that include simultaneous movements of multiple facial landmarks relative to one another, the combination of facial tracking inputs corresponding to the predefined expression;
- generating a graphical depiction of the user based on the graphical element that corresponds with the movements and the positions of the facial landmarks relative to one another;
- and
- overlaying the graphical depiction of the user at a position upon the image data at the client device.
10 more elements
A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising:
- causing display of image data at a client device, the image data comprising a set of facial landmarks;
- detecting the set of facial landmarks within the image data;
- and
- responsive to the detecting the set of facial landmarks within the image data, performing operations comprising:
- identifying a user profile based on attributes of the set of facial landmarks from within the image data;
- accessing user profile data associated with the user profile, the user profile data including an expression definition that maps facial tracking inputs that comprise a landmark identifier, an input value indicating movement relative to a static position, and a movement direction to a predefined expression that includes a corresponding graphical element;
- determining an expression of a user based on a combination of facial tracking inputs that include simultaneous movements of multiple facial landmarks relative to one another, the combination of facial tracking inputs corresponding to the predefined expression;
- generating a graphical depiction of the user based on the graphical element that corresponds with the movements and the positions of the facial landmarks relative to one another;
- and
- overlaying the graphical depiction of the user at a position upon the image data at the client device.
7 more elements
Dependencies are inferred from the claim text.