Telepresence of users in interactive virtual spaces

A system for rendering three-dimensional virtual scenes, the system comprising:

• a communication interface that communicates with a first user device over a communication network to:
• receive a first visual dataset corresponding to a three-dimensional shape of a first user in a real-world environment, the first visual dataset captured by the first user device;
• and
• receive sensor data including one or more measurements taken by one or more sensors in the real-world environment;
• and
• a processor coupled to memory storing instructions, wherein the processor executes the instructions stored in memory to:
• render a three-dimensional virtual scene that includes a first three-dimensional avatar and one or more virtual objects, the first three-dimensional avatar rendered based on the first visual dataset;
• identify a trigger condition when the first three-dimensional avatar is detected interacting with at least one of the virtual objects, wherein identifying the trigger condition includes tracking boundaries of the first three-dimensional avatar and boundaries of the at least one virtual object;
• and
• generate a visual overlay that displays the received measurements, the generated overlay appearing within the three-dimensional virtual scene that is updated based on the iidentified trigger condition associated with the detected interaction by the first three-dimensional avatar with the at least one virtual object, wherein communication interface transmits the updated three-dimensional virtual scene to a second user device, the transmitted three-dimensional virtual scene including the first three-dimensional avatar interacting with the at least one virtual object and the visual overlay that displays the measurements.

A non-transitory computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for rendering three-dimensional virtual scenes, the method comprising:

• receiving a first visual dataset corresponding to a three-dimensional shape of a first user in a real-world environment, the first visual dataset captured by a first user device;
• rendering a three-dimensional virtual scene that includes a first three-dimensional avatar and one or more virtual objects, the first three-dimensional avatar rendered based on the first visual dataset;
• receiving sensor data including one or more measurements taken by one or more sensors in the real-world environment of the first user;
• identifying a trigger condition when the first three-dimensional avatar is detected interacting with at least one of the virtual objects, wherein identifying the trigger condition includes tracking boundaries of the first three-dimensional avatar and boundaries of the at least one virtual object;
• generating a visual overlay that displays the received measurements, the generated overlay appearing within the three-dimensional virtual scene that is updated based on the identified trigger condition associated with the detected interaction by the first three-dimensional avatar with the at least one virtual object;
• and
• transmitting the updated three-dimensional virtual scene to a second user device, the transmitted three-dimensional virtual scene including the first three-dimensional avatar interacting with the at least one virtual object and the visual overlay that displays the measurements.