# US 9,519,989 B2 · Visual representation expression based on player expression > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,989 B2 - Document ID: US9519989B2 - Status: Granted - Kind: B2 - Application number: 13/784,480 - Filed: 2013-03-04 - Published: 2013-07-25 - Granted: 2016-12-13 - Earliest claimed: 2009-07-09 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Kathryn Stone Perez; Alex Kipman; Nicholas D. Burton; Andrew Wilson - Web page: https://patentgrep.com/patent/US9519989B2 ## CPC - G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites - A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) - A63F 13/42: by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Using facial recognition and gesture/body posture recognition techniques, a system can naturally convey the emotions and attitudes of a user via the user's visual representation. Techniques may comprise customizing a visual representation of a user based on detectable characteristics, deducting a user's temperament from the detectable characteristics, and applying attributes indicative of the temperament to the visual representation in real time. Techniques may also comprise processing changes to the user's characteristics in the physical space and updating the visual representation in real time. For example, the system may track a user's facial expressions and body movements to identify a temperament and then apply attributes indicative of that temperament to the visual representation. Thus, a visual representation of a user, which may be, for example, an avatar or fanciful character, can reflect the user's expressions and moods in real time. ## Claims ### Claim 1 A method for applying attributes indicative of a user's temperament to a visual representation of the user, the method comprising: rendering a visual representation of a user; receiving, at a computing system, first data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game; analyzing, at the computing system, the first data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user's temperament; determining, at the computing system, a pre-existing animation to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and displaying, at the computing system, the pre-existing animation for the visual representation on a display device. ### Claim 2 (depends on 1) The method of claim 1, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model. ### Claim 3 (depends on 1) The method of claim 1, wherein analyzing the first data of the user in the physical space comprises: analyzing a user's characteristics, a user's physical features, a user's behavior, a user's speech pattern, a user's voice, a gesture, or history data. ### Claim 4 (depends on 1) The method of claim 1, wherein the first data indicative of the user in the physical space is representative of at least one of a user's characteristics in the physical space. ### Claim 5 (depends on 1) The method of claim 1, further comprising: applying at least one of a user's characteristics to the visual representation. ### Claim 6 (depends on 1) The method of claim 1, wherein receiving first data indicative of the user in the physical space comprises: receiving data via a capture device. ### Claim 7 (depends on 1) The method of claim 1, wherein the deduced emotional temperament comprises generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry. ### Claim 8 (depends on 1) The method of claim 1, further comprising: receiving a second data indicative of the user in the physical space; determining a change to the user's emotional temperament based on a difference between the second data and the first data; determining a second pre-existing animation based on the change to the user's emotional temperament, the second pre-existing animation having a motion, pose or facial expression not expressed in the second data; and displaying the second pre-existing animation for the visual representation on the display device. ### Claim 9 (depends on 1) The method of claim 1, further comprising: selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the deduced emotional temperament. ### Claim 10 A system for applying attributes indicative of a user's temperament to a visual representation, the system comprising: a processor; and memory communicatively coupled with the processor, the memory bearing processor-executable instructions that, upon execution on the processor, cause the system to at least: render a visual representation of a user; receive data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game; analyze the data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user's temperament; determine a pre-existing animation to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and display the pre-existing animation for the visual representation on a display device. ### Claim 11 (depends on 10) The system of claim 10, further comprising processor-executable instructions that cause the system to at least: display the pre-existing animation for the visual representation on a display device in real time with respect to the receipt of data indicative of the user in the physical space. ### Claim 12 (depends on 11) The system of claim 11, the memory having stored thereon data correlating characteristics to a particular emotional temperament. ### Claim 13 A computer-readable storage device that is not only a propagating signal comprising computer-executable instructions that, upon execution on a computer, cause the computer to perform operations comprising: rendering a visual representation of a user; receiving data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game; analyzing the first data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user's temperament; determining a pre-existing animation of to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and displaying the pre-existing animation for the visual representation on a display device. ### Claim 14 (depends on 13) The computer-readable storage device of claim 13, wherein displaying the pre-existing animation for the visual representation on a display device is performed in real time with respect to receiving data indicative of the user in the physical space. ### Claim 15 (depends on 13) The computer-readable storage device of claim 13, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model. ### Claim 16 (depends on 13) The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising: applying at least one of a user's characteristics to the visual representation, the data comprising data representative of the user's characteristics. ### Claim 17 (depends on 13) The computer-readable storage device of claim 13, wherein receiving data indicative of the user in the physical space comprises: receiving data via a capture device. ### Claim 18 (depends on 13) The computer-readable storage device of claim 13, wherein the deduced emotional temperament comprises at least one of generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry. ### Claim 19 (depends on 13) The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising: providing a filter representing an emotional temperament identity gesture, the filter comprising base information about the emotional temperament identity gesture; and applying the filter to the first data and determining an output from the base information about the emotional temperament identity gesture. ### Claim 20 (depends on 13) The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising: selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the deduced temperament.