# US 9,436,276 B2 · Second-person avatars > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,276 B2 - Document ID: US9436276B2 - Status: Granted - Kind: B2 - Application number: 12/392,538 - Filed: 2009-02-25 - Published: 2010-08-26 - Granted: 2016-09-06 - Earliest claimed: 2009-02-25 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Eyal Ofek; Blaise H. Aguera y Arcas; Avi Bar-Zeev; Gur Kimchi; Jason Szabo - Inventors: Eyal Ofek; Blaise H. Aguera y Arcas; Avi Bar-Zeev; Gur Kimchi; Jason Szabo - Web page: https://patentgrep.com/patent/US9436276B2 ## CPC - A63F 13/12 - G06T 13/40: of characters, e.g. humans, animals or virtual beings - A63F 13/30: Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers - A63F 13/537: using indicators, e.g. showing the condition of a game character on screen - A63F 13/69: by enabling or updating specific game elements, e.g. unlocking hidden features, items, levels or versions - A63F 13/79: involving player-related data, e.g. identities, accounts, preferences or play histories - G02B 27/01: Head-up displays - G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 9/453: {Help systems} - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - A63F 2300/301: using an additional display connected to the game console, e.g. on the controller - A63F 2300/305: for providing a graphical or textual hint to the player - A63F 2300/8082: Virtual reality - G02B 2027/0138: {comprising image capture systems, e.g. camera} - G02B 2027/014: {comprising information/image processing systems} - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The claimed subject matter relates to an architecture that can provide for a second-person avatar. The second-person avatar can rely upon a second-person-based perspective such that the avatar is displayed to appear to encompass all or portions of a target user. Accordingly, actions or a configuration of the avatar can serve as a model or demonstration for the user in order to aid the user in accomplishing a particular task. Updates to avatar activity or configuration can be provided by a dynamic virtual handbook. The virtual handbook can be constructed based upon a set of instruction associated with accomplishing the desired task and further based upon features or aspects of the user as well as those of the local environment. ## Claims ### Claim 1 A computer implemented system that provides a second-person avatar in order to facilitate manual instruction, comprising: a user-interface component that includes a projector that displays at least a portion of the second-person avatar in a volume of space that encapsulates at least a portion of a target user; an interpretation component that translates a set of instructions from a text-based instruction manual into a virtual handbook of avatar actions for the second-person avatar to perform a task, wherein at least one instruction of the set of instructions is translated based upon one or more physical features of the target user identified in an image of the target user; and an illustration component that: modifies the second-person avatar to reflect the one or more physical features of the target user, and illustrates the second-person avatar performing the avatar actions in a sequence to complete the task while the user-interface component tracks the target user. ### Claim 2 (depends on 1) The system of claim 1, wherein the user-interface component further includes at least one of a camera, a speaker, a display, a keyboard, a network adapter, a microphone, an accelerometer, a biometric sensor, and a gaze-tracking sensor. ### Claim 3 (depends on 1) The system of claim 1, wherein the user-interface component is included in a headset or wearable unit. ### Claim 4 (depends on 1) The system of claim 1, wherein the second-person avatar is visible when viewed through a head up display or a virtual retinal display included in the user-interface component. ### Claim 5 (depends on 1) The system of claim 1, wherein the illustration component identifies a deviation when a user action or configuration deviates from the avatar actions beyond an acceptable error threshold. ### Claim 6 (depends on 5) The system of claim 5, wherein the illustration component constructs a deviation grid that identifies a section of the second-person avatar that deviates from an associated portion of the target user; and the user-interface updates a color of the section of the second-person avatar based upon the deviation grid. ### Claim 7 (depends on 5) The system of claim 5, wherein the illustration component constructs a speech report that explains the deviation or a corrective process thereof in verbal form; and the user-interface outputs the speech report. ### Claim 8 (depends on 1) The system of claim 1, wherein the virtual handbook identifies a disparate, physical feature or object included in a local, real-world environment that pertains to accomplishing the task; and the user-interface visually highlights the physical feature or object. ### Claim 9 (depends on 1) The system of claim 1, wherein the interpretation component receives the set of instructions that relate to accomplishing the task and translates the set of instructions into the avatar actions that compose the virtual handbook. ### Claim 10 (depends on 9) The system of claim 9, wherein the interpretation component translates the set of instructions into the virtual handbook based upon a feature associated with the target user or a profile thereof. ### Claim 11 (depends on 10) The system of claim 10, wherein the feature associated with the target user or the profile thereof relates to: (1) a size or dimension of the target user and (2) a handedness of the target user. ### Claim 12 (depends on 9) The system of claim 9, wherein the interpretation component includes an object recognition component that aids in translating the set of instructions into the virtual handbook based upon a feature or an object associated with a local, real-world environment. ### Claim 13 (depends on 9) The system of claim 9, wherein the interpretation component includes a language recognition component that aids in translating the set of instructions into the virtual handbook based upon textual or verbal inputs. ### Claim 14 (depends on 9) The system of claim 9, wherein the interpretation component receives the set of instructions from a remote expert by way of a live or recorded manual demonstration of accomplishing the task in a remote environment, or based upon live or recorded verbal instructions. ### Claim 15 (depends on 9) The system of claim 9, wherein the interpretation component receives the set of instructions from an instruction manual or guide. ### Claim 16 (depends on 9) The system of claim 9, wherein the interpretation component receives a verbal query from the target user, examines a state of the target user, and updates the virtual handbook based upon the query and the state. ### Claim 17 A method of providing a second-person avatar in order to facilitate manual instruction, comprising: displaying at least a portion of the second-person avatar in a volume of space that encapsulates at least a portion of a target user; translating a set of instructions from a text-based instruction manual into a virtual handbook of avatar actions for the second-person avatar to perform a task, wherein at least one instruction of the set of instructions is translated based upon one or more physical features of the target user identified in an image of the target user; modifying the second-person avatar to reflect the one or more physical features of the target user; and illustrating the second-person avatar performing the avatar actions in a sequence to complete the task while the user-interface component tracks the target user. ### Claim 18 (depends on 17) The method of claim 17, wherein the second-person avatar is visible when viewed through a head up display or a virtual retinal display. ### Claim 19 One or more hardware memory devices having embodied thereon computer-usable instructions which, when executed by one or more processing devices, facilitate a method of providing a second-person avatar in order to facilitate manual instruction, the method comprising: displaying at least a portion of the second-person avatar in a volume of space that encapsulates at least a portion of a target user; translating a set of instructions from a text-based instruction manual into a virtual handbook of avatar actions for the second-person avatar to perform a task, wherein at least one instruction of the set of instructions is translated based upon one or more physical features of the target user identified in an image of the target user; modifying the second-person avatar to reflect the one or more physical features of the target user; and illustrating the second-person avatar performing the avatar actions in a sequence to complete the task while the user-interface component tracks the target user. ### Claim 20 (depends on 19) The one or more hardware memory devices of claim 19, wherein the second-person avatar is visible when viewed through a head up display or a virtual retinal display.