# US 2018/0027307 A1 · Emotional reaction sharing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2018/0027307 A1 - Document ID: US20180027307A1 - Status: Published application - Kind: A1 - Application number: 15/218,268 - Filed: 2016-07-25 - Published: 2018-01-25 - Earliest claimed: 2016-07-25 - Assignees: Yahoo! Inc. - Applicants: Yahoo!, Inc. - Inventors: Bin Ni; Gregory Davis Choi; Adam Bryan Mathes - Web page: https://patentgrep.com/patent/US20180027307A1 ## CPC - G06T 13/205: {driven by audio data} - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G06V 40/165: {using facial parts and geometric relationships} - G06V 40/176: {Dynamic expression} - H04N 21/42203: {sound input device, e.g. microphone} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)} - H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} - G06T 2200/08: involving all processing steps from image acquisition to 3D model generation - G06T 2207/10016: Video; Image sequence - G06T 2207/30201: Face - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract One or more computing devices, systems, and/or methods for emotional reaction sharing are provided. For example, a client device captures video of a user viewing content, such as a live stream video. Landmark points, corresponding to facial features of the user, are identified and provided to a user reaction distribution service that evaluates the landmark points to identify a facial expression of the user, such as a crying facial expression. The facial expression, such as landmark points that can be applied to a three-dimensional model of an avatar to recreate the facial expression, are provided to client devices of users viewing the content, such as a second client device. The second client device applies the landmark points of the facial expression to a bone structure mapping and a muscle movement mapping to create an expressive avatar having the facial expression for display to a second user. ## Claims ### Claim 1 A method of emotional reaction sharing, the method involving a computing device comprising a processor, and the method comprising: executing, on the processor, instructions that cause the computing device to perform operations, the operations comprising: responsive to determining that a user is viewing content through a client device, initializing a camera of the client device to capture one or more frames of video of the user; evaluating a first frame of the video to identify a set of facial features of the user; generating a set of landmark points, within the first frame, representing the set of facial features; and sending the set of landmark points to a user reaction distribution service for identifying a facial expression of the user, based upon the set of landmark points, for display through a second client device to a second user. ### Claim 2 (depends on 1) The method of claim 1, wherein the set of landmark points comprise coordinates of between about 4 landmark points to about 240 landmark points, a landmark point specifying a location of a facial feature. ### Claim 3 (depends on 1) The method of claim 1, wherein the set of landmark points comprise a number of landmark points that is less than one percent of a pixel count of the first frame. ### Claim 4 (depends on 1) The method of claim 1, wherein the facial expression is constructed upon an avatar, representing the user, for display through the second client device. ### Claim 5 (depends on 1) The method of claim 1, comprising: initializing a microphone of the client device to capture audio content of the user; evaluating the audio content to identify a mood of the user; and utilizing the mood of the user for identifying the facial expression of the user. ### Claim 6 (depends on 1) The method of claim 1, wherein the second user is viewing the content. ### Claim 7 (depends on 1) The method of claim 1, comprising: responsive to determining that the user is not viewing the content, transitioning the camera into a low power state. ### Claim 8 A computing device comprising: a processor; and memory comprising processor-executable instructions that when executed by the processor cause performance of operations, the operations comprising: receiving a first set of landmark points and a second set of landmark points from a client device, the first set of landmark points representing a set of facial features of a user at a first point in time and the second set of landmark points representing the set of facial features of the user at a second point in time while the user is viewing content through the client device; evaluating the first set of landmark points and the second set of landmark points, using a facial expression recognition algorithm that maps changes in location of landmark points to facial movements indicative of facial expressions, to identify a facial expression of the user while the user is viewing the content; and sending the facial expression to a second client device, of a second user that is viewing the content, for display to the second user. ### Claim 9 (depends on 8) The computing device of claim 8, wherein the operations comprise: identifying a set of facial expressions of users view the content based upon landmark points received from client devices of the users; ranking the set of facial expressions based upon an interestingness expression metric; and selecting the facial expression, from the set of facial expressions, to send based upon the facial expression having a ranking exceeding a threshold. ### Claim 10 (depends on 8) The computing device of claim 8, wherein the operations comprise: identifying a set of facial expressions of users viewing the content based upon landmark points received from client devices of the users; ranking the set of facial expressions based upon a common expression metric; and selecting the facial expression, from the set of facial expressions, to send based upon the facial expression having a ranking exceeding a threshold. ### Claim 11 (depends on 8) The computing device of claim 8, wherein the operations comprise: training the facial expression recognition algorithm to identify a type of facial expression based upon a training set of landmark points associated with the type of facial expression, the training set of landmark points corresponding to facial movement indicative of the type of facial expression. ### Claim 12 (depends on 8) The computing device of claim 8, wherein the facial expression is identified and sent to the second client device in real-time during viewing of the content by the user and the second user. ### Claim 13 A non-transitory machine readable medium having stored thereon processor-executable instructions that when executed cause performance of operations, the operations comprising: receiving, at a client device of a user viewing content, a facial expression, of a second user while viewing the content, from a user reaction distribution service; applying the facial expression to a three-dimensional model of an avatar to generate an expressive avatar having the facial expression; and displaying the expressive avatar to the user while the user views the content. ### Claim 14 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the displaying comprises: displaying the expressive avatar within a user interface element; and applying an animation display property to the user interface element, the animation display property specifying that the user interface element is to expand in size until reaching a size threshold and then the user interface element is to disappear. ### Claim 15 (depends on 13) The non-transitory machine readable medium of claim 13, comprising: selecting the avatar from an avatar repository based upon a user avatar preference. ### Claim 16 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the content is at least one of a video conference, a video, a videogame, augmented reality, or a live stream video. ### Claim 17 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the operations comprise: evaluating the content to identify a visual feature of the content; identifying a region within the content into which the expressive avatar can be added without occluding the visual feature above a threshold amount; and displaying the expressive avatar within the region. ### Claim 18 (depends on 17) The non-transitory machine readable medium of claim 17, wherein the operations comprise: modifying at least one of the content or the expressive avatar to reduce occlusion. ### Claim 19 (depends on 13) The non-transitory machine readable medium of claim 13, wherein the facial expression comprises a set of landmark points corresponding to a set of facial features of the second user, and the operations comprising: applying the set of landmark points to the three-dimensional model. ### Claim 20 (depends on 19) The non-transitory machine readable medium of claim 19, wherein the applying the set of landmark points comprises: selecting a bone structure mapping from a set of bone structure mappings based upon the set of landmark points; selecting a muscle movement mapping from a set of muscle movement mappings based upon the set of landmark points; and applying the bone structure mapping and the muscle movement mapping to the three-dimensional model to create the expressive avatar.