# US 12,192,257 B2 · 2D and 3D transitions for renderings of users participating in communication sessions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,257 B2 - Document ID: US12192257B2 - Status: Granted - Kind: B2 - Application number: 17/824,869 - Filed: 2022-05-25 - Published: 2023-11-30 - Granted: 2025-01-07 - Earliest claimed: 2022-05-25 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Jason Thomas Faulkner; Mansoor Jafry; Xonatia Ravelle Lee; Chad Aron Voss; Albert Robles; Timothy James Brookins - Web page: https://patentgrep.com/patent/US12192257B2 ## CPC - H04L 65/403: Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) - H04N 7/157: {defining a virtual conference space and using avatars or agents (computer conference optimisation or adaptation H04L12/1827)} - H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission} - H04N 13/366: using viewer tracking - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Systems for transitioning a user interface arrangement from a display of a two-dimensional image of a user to a rendering of a three-dimensional representation of the user is provided. A system can start with a UI including a rendering of a user that is based on a 2D image file. The system can receive an input that is configured to cause the system to transition the display of the rendering of the 2D image of the select user to a rendering of the three-dimensional representation of the select user. To display the rendering of the 3D representation of the select user, the system uses permission data and a three-dimensional model defining a position and orientation to display the 3D representation of the user. The system allows users to switch between viewing modes to allow users to interact with content using the most effective type of hardware. ## Claims ### Claim 1 A method for transitioning a user interface from a rendering of a two-dimensional image of a user to a rendering of a three-dimensional representation of the user while the user is participating in a communication session, the method configured for execution on a system, the method comprising: receiving a live video stream from an image video camera directed toward the user for the generation of rendering of the two-dimensional image of the user within a two-dimensional 2D environment; providing the two-dimensional environment for the communication session and a three-dimensional environment for the communication session, the three-dimensional environment including virtual objects that are positioned based on coordinates; causing a display of the user interface comprising the rendering of the two-dimensional image of the user, wherein the rendering of the two-dimensional image is generated from the live video stream of the camera directed toward the user, wherein the user interface concurrently displays the rendering of the two-dimensional image of the user with renderings of other users participating in the communication session with the user; during the communication session, receiving an input for causing a transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user; and in response to the input for causing the transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user: accessing a three-dimensional model defining a position and orientation of the three-dimensional representation of the user within the three-dimensional environment, and removing the rendering of the two-dimensional image of the user from the two-dimensional environment, and adding the display of the three-dimensional representation of the user in a rendering of the three-dimensional environment using the position and the orientation defined in the three-dimensional model. ### Claim 2 (depends on 1) The method of claim 1, wherein the input for causing the transition includes input data indicating that the user is interacting with a computing device having a keyboard and mouse, wherein permission data is modified to allow the transition based on a detection of the keyboard and mouse in communication of a client computing device associated with the user, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the keyboard and mouse in communication with the client computing device. ### Claim 3 (depends on 1) The method of claim 1, wherein the input for causing the transition includes input data indicating that the user is interacting with a desktop computing device having a keyboard, wherein permission data is modified to allow the transition based on a detection of the keyboard in communication of the desktop computing device associated with the user, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the keyboard in communication with the desktop computing device associated with the user. ### Claim 4 (depends on 1) The method of claim 1, wherein the position and orientation of the three-dimensional representation is configured to provide a display of the representation such that the representation is appearing to look at a rendering of shared content in response to determining that a threshold number of representations of other users participating in the communication session are positioned to appear to be looking at the rendering of the shared content. ### Claim 5 (depends on 1) The method of claim 1, wherein the input for causing the transition includes input data indicating that the user has put on a head-mounted display device for use in the communication session, wherein the input data is generated from a motion sensor mounted to the head-mounted display device, the sensor detecting movement of the user indicating that the user has put on a head-mounted display device. ### Claim 6 A system for transitioning a user interface from a rendering of a two-dimensional image of a user to a rendering of a three-dimensional representation of the user while the user is participating in a communication session, the system comprising: one or more processing units; and a computer-readable storage medium having encoded thereon computer-executable instructions to cause the one or more processing units to perform a method comprising: receiving a live video stream from an image video camera directed toward the user for the generation of rendering of the two-dimensional image of the user within a two-dimensional 2D environment; providing the two-dimensional environment for the communication session and a three-dimensional environment for the communication session, the three-dimensional environment including virtual objects that are positioned based on coordinates; causing a display of the user interface comprising the rendering of the two-dimensional image of the user, wherein the rendering of the two-dimensional image is generated from the live video stream of the camera directed toward the user, wherein the user interface concurrently displays the rendering of the two-dimensional image of the user with renderings of other users participating in the communication session with the user; during the communication session, receiving an input for causing a transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user; and in response to the input for causing the transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user: accessing a three-dimensional model defining a position and orientation of the three-dimensional representation of the user within the three-dimensional environment, and removing the rendering of the two-dimensional image of the user from the two-dimensional environment, and adding the display of the three-dimensional representation of the user in a rendering of the three-dimensional environment using the position and the orientation defined in the three-dimensional model. ### Claim 7 (depends on 6) The system of claim 6, wherein the input for causing the transition includes input data indicating that the user is interacting with a computing device having a keyboard, wherein permission data is modified to allow the transition based on a detection of the keyboard in communication of a client computing device associated with the user, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the keyboard in communication with the client computing device associated with the user. ### Claim 8 (depends on 6) The system of claim 6, wherein the position and orientation of the three-dimensional representation is configured to provide a display of the representation such that the representation is appearing to look at a rendering of shared content in response to determining that a threshold number of representations of other users participating in the communication session are positioned to appear to be looking at the rendering of the shared content. ### Claim 9 (depends on 6) The system of claim 6, wherein the position and orientation of the three-dimensional representation is configured to provide a display of the representation such that the representation is appearing to look at the representations of other users participating in the communication session, in response to determining that a threshold number of representations of other users participating in the communication session are positioned and oriented to appear to be looking at one another. ### Claim 10 (depends on 6) The system of claim 6, wherein the input for causing the transition includes input data indicating that the user has put on a head-mounted display device for use in the communication session, wherein the input data is generated from a motion sensor mounted to the head-mounted display device, the sensor detecting movement of the user indicating that the user has put on a head-mounted display device. ### Claim 11 (depends on 6) The system of claim 6, wherein the input for causing the transition includes input data indicating a predetermined device type, wherein permission data is modified to allow the transition based on a detection of the predetermined device type being a desktop computing device, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the detection of the predetermined device type. ### Claim 12 A computer-readable storage medium having encoded thereon computer-executable instructions to cause one or more processing units of a system to perform a method for transitioning a user interface from a rendering of a two-dimensional image of a user to a rendering of a three-dimensional representation of the user while the user is participating in a communication session, the method comprising: receiving a live video stream from an image video camera directed toward the user for the generation of rendering of the two-dimensional image of the user within a two-dimensional 2D environment; providing the two-dimensional environment for the communication session and a three-dimensional environment for the communication session, the three-dimensional environment including virtual objects that are positioned based on coordinates; causing a display of the user interface comprising the rendering of the two-dimensional image of the user, wherein the rendering of the two-dimensional image is generated from the live video stream of the camera directed toward the user, wherein the user interface concurrently displays the rendering of the two-dimensional image of the user with renderings of other users participating in the communication session with the user; during the communication session, receiving an input for causing a transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user; and in response to the input for causing the transition of the display of the rendering of the two-dimensional image of the user to the rendering of the three-dimensional representation of the user: accessing a three-dimensional model defining a position and orientation of the three-dimensional representation of the user within the three-dimensional environment, and removing the rendering of the two-dimensional image of the user from the two-dimensional environment, and adding the display of the three-dimensional representation of the user in a rendering of the three-dimensional environment using the position and the orientation defined in the three-dimensional model. ### Claim 13 (depends on 12) The computer-readable storage medium of claim 12, wherein the input for causing the transition includes input data indicating that the user is interacting with a computing device having a keyboard and mouse, wherein permission data is modified to allow the transition based on a detection of the keyboard and mouse in communication of a client computing device associated with the user, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the keyboard and mouse in communication with the client computing device. ### Claim 14 (depends on 12) The computer-readable storage medium of claim 12, wherein the input for causing the transition includes input data indicating that the user is interacting with a computing device having a keyboard, wherein permission data is modified to allow the transition based on a detection of the keyboard in communication of a client computing device associated with the user, wherein the system is configured to only allow the transition of the user interface in response to determining that the permission data allows the transition based on the detection of the keyboard in communication with the client computing device associated with the user. ### Claim 15 (depends on 12) The computer-readable storage medium of claim 12, wherein the position and orientation of the three-dimensional representation is configured to provide a display of the representation such that the representation is appearing to look at a rendering of shared content in response to determining that a threshold number of representations of other users participating in the communication session are positioned to appear to be looking at the rendering of the shared content. ### Claim 16 (depends on 12) The computer-readable storage medium of claim 12, wherein the position and orientation of the three-dimensional representation is configured to provide a display of the representation such that the representation is appearing to look at the representations of other users participating in the communication session, in response to determining that a threshold number of representations of other users participating in the communication session are positioned and oriented to appear to be looking at one another. ### Claim 17 (depends on 12) The computer-readable storage medium of claim 12, wherein the input for causing the transition includes input data indicating that the user has put on a head-mounted display device for use in the communication session, wherein the input data is generated from a motion sensor mounted to the head-mounted display device, the sensor detecting movement of the user indicating that the user has put on a head-mounted display device. ### Claim 18 (depends on 1) The method of claim 1, wherein the renderings of other users participating in the communication session are maintained in the user interface during the modification that removes the rendering of the two-dimensional image of the user and adds the three-dimensional representation of the user. ### Claim 19 (depends on 1) The method of claim 1, wherein the user interface comprises a 2D region and a 3D region, wherein the modification of the user interface arrangement removes the rendering of the two-dimensional image of the user from the 2D region and adds the three-dimensional representation of the user within the 3D region. ### Claim 20 (depends on 1) The method of claim 1, wherein the user interface concurrently displays the two-dimensional environment and the three-dimensional environment.