# US 11,910,132 B2 · Head tracking for video communications in a virtual environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,910,132 B2 - Document ID: US11910132B2 - Status: Granted - Kind: B2 - Application number: 17/515,494 - Filed: 2021-10-31 - Published: 2023-05-04 - Granted: 2024-02-20 - Earliest claimed: 2021-10-31 - Assignees: Zoom Video Communications, Inc. - Applicants: Zoom Video Communications, Inc. - Inventors: Jordan Thiel - Web page: https://patentgrep.com/patent/US11910132B2 ## CPC - H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)} - G03H 1/0005: {Adaptation of holography to specific applications (holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042)} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/014: {Hand-worn input/output arrangements, e.g. data gloves} - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G06T 15/20: Perspective computation - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G03H 2001/0088: {for video-holography, i.e. integrating hologram acquisition, transmission and display} - G06T 2215/16: Using real world measurements to influence rendering - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G03H: HOLOGRAPHIC PROCESSES OR APPARATUS (holograms, e.g. point holograms, used as ordinary optical elements G02B5/32; producing stereoscopic or other three-dimensional effects G02B30/00; diffraction-grating systems G02B27/44; systems using moiré fringes G02B27/60; optical logic elements G02F3/00; stereo-photography G03B35/00; photosensitive materials or processes for photographic purposes G03C; {stereo-photographic or similar processes G03C9/00}; apparatus for processing exposed photographic materials G03D; analogue computers performing mathematical operations with the aid of optical elements G06E3/00; authentication by radiation, of concealed information carried by holograms or diffraction gratings G06K19/16; holographic storage G11B7/0065, G11C13/04; {stereoscopic or other three dimensional effects in television systems H04N13/00}) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods, systems, and apparatus, including computer programs encoded on computer storage media relate to a method for head tracking for video communications in a virtual environment. The system may provide a video conference session in a virtual environment. The system may provide a digital representation of the video conference participant in the virtual environment. The system may display one or more views of the virtual environment in the video conference. The system may track movement of the video conference participant to generate user movement information and may display movement of the video conference participant on the digital representation. ## Claims ### Claim 1 A method comprising: providing a video conference session in a virtual environment; providing a digital representation of a video conference participant in the virtual environment and capturing a left view of the virtual environment to the left of the digital representation and a right view of the virtual environment to the right of the digital representation; providing the captured left view for display on a left portion of one or more displays and the captured right view for display on a right portion of the one or more displays; receiving a video stream of the video conference participant; and tracking, in the video stream, head movement of the video conference participant to generate head movement information comprising: detecting, within the video stream, head movement that corresponds with the video conference participant performing an act of viewing past a known physical boundary; generating head movement information that includes an indication of the act of viewing past the known physical boundary; and triggering an action for updating the digital representation based on the indication. ### Claim 2 (depends on 1) The method of claim 1, wherein the virtual environment comprises one of: (i) a virtual reality (VR) environment including 3D avatars of one or more users and (ii) an augmented reality (AR) environment comprising one or more AR holograms. ### Claim 3 (depends on 1) The method of claim 1, further comprising: providing the captured left view of the virtual environment and the captured right view of the virtual environment as part of the video stream or in a second video stream in the video conference session. ### Claim 4 (depends on 1) The method of claim 1, further comprising: turning a head of the digital representation of the video conference participant by an amount corresponding to an amount of a turn of the head of the video conference participant. ### Claim 5 (depends on 1) The method of claim 1, further comprising: turning a body of the digital representation of the video conference participant based on the head movement information. ### Claim 6 (depends on 1) The method of claim 1, wherein the digital representation of the video conference participant comprises a flat shape in the virtual environment displaying at least a portion of the video stream of the video conference participant. ### Claim 7 A non-transitory computer readable medium that stores executable program instructions that when executed by one or more computing devices configure the one or more computing devices to perform operations comprising: providing a video conference session in a virtual environment; providing a digital representation of a video conference participant in the virtual environment and capturing a left view of the virtual environment to the left of the digital representation and a right view of the virtual environment to the right of the digital representation; providing the captured left view for display on a left portion of one or more displays and the captured right view for display on a right portion of the one or more displays; receiving a video stream of the video conference participant; and tracking, in the video stream, head movement of the video conference participant to generate head movement information comprising: detecting, within the video stream, head movement that corresponds with the video conference participant performing an act of viewing past a known physical boundary; generating head movement information that includes an indication of the act of viewing past the known physical boundary; and triggering an action for updating the digital representation based on the indication. ### Claim 8 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the virtual environment comprises one of: (i) a virtual reality (VR) environment including 3D avatars of one or more users and (ii) an augmented reality (AR) environment comprising one or more AR holograms. ### Claim 9 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the executable program instructions further configure the one or more computing devices to perform operations comprising: providing the captured left view of the virtual environment and the captured right view of the virtual environment as part of the video stream or in a second video stream in the video conference session. ### Claim 10 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the executable program instructions further configure the one or more computing devices to perform operations comprising: turning a head of the digital representation of the video conference participant by an amount corresponding to an amount of a turn of the head of the video conference participant. ### Claim 11 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the executable program instructions further configure the one or more computing devices to perform operations comprising: turning a body of the digital representation of the video conference participant based on detected head movement information. ### Claim 12 (depends on 7) The non-transitory computer readable medium of claim 7, wherein the digital representation of the video conference participant comprises a flat shape in the virtual environment displaying at least a portion of the video stream of the video conference participant. ### Claim 13 A system comprising one or more processors configured to perform operations of: providing a video conference session in a virtual environment; providing a digital representation of a video conference participant m the virtual environment and capturing a left view of the virtual environment to the left of the digital representation and a right view of the virtual environment to the right of the digital representation; providing the captured left view for display on a left portion of one or more displays and the captured right view for display on a right portion of the one or more displays; receiving a video stream of the video conference participant; and tracking, in the video stream, head movement of the video conference participant to generate head movement information comprising: detecting, within the video stream, head movement that corresponds with the video conference participant performing an act of viewing past a known physical boundary; generating head movement information that includes an indication of the act of viewing past the known physical boundary; and triggering an action for updating the digital representation based on the indication. ### Claim 14 (depends on 13) The system of claim 13, wherein the virtual environment comprises a VR environment including 3D avatars of one or more users. ### Claim 15 (depends on 13) The system of claim 13, wherein the processors are further configured to perform the operations of: turning a head of the digital representation of the video conference participant by an amount corresponding to a turn of the head of the video conference participant. ### Claim 16 (depends on 13) The system of claim 13, wherein the processors are further configured to perform the operations of: turning a body of the digital representation of the video conference participant based on detected head movement information. ### Claim 17 (depends on 1) The method of claim 1, wherein detecting head movement that corresponds with the video conference participant performing an act of viewing past a known physical boundary comprises: identifying a direction of movement of the act; and wherein generating head movement information that includes an indication of the act of viewing past the known physical boundary comprises: associating a direction attribute with the head movement information based on the identified direction of movement of the act. ### Claim 18 (depends on 1) The method of claim 1, wherein triggering an action for updating the digital representation comprises: updating a current viewpoint of the digital representation based at least on a direction attribute of the indication of the act of viewing past the known physical boundary, wherein the known physical boundary comprises: a current physical placement of an edge relative to a screen displaying at least a portion of a respective view of the virtual environment. ### Claim 19 (depends on 1) The method of claim 1, wherein the left view of the virtual environment to the left of the digital representation is a different view than the right view of the virtual environment to the right of the digital representation.