Three-dimensional layered conferencing system with distributed computing for high-fidelity rendering of virtual environments and method

A non-transitory computer readable storage medium wherein at least one instruction is stored in the medium, and the at least one instruction is loaded and executed by a processor to provide a three-dimensional layered conferencing system with distributed computing for high-fidelity rendering of a virtual environment comprising:

• a high-fidelity layer rendered remotely on a server and streamed to a user device in approximately real-time over a network without intended delay comprising the virtual environment in three dimensions and in high fidelity for a communication session;
• a local layer rendered locally on the user device, which is separate and distinct from rendering of the virtual environment by the server, comprising an avatar for a user to be integrated into the virtual environment for the communication session, the local layer being synchronized with high-fidelity layer that is independently rendered on a server and streamed to the user device over the network;
• a communication layer operated over a decentralized peer-to-peer network comprising audiovisual content of the user communicated in approximately real-time without intended delay for the communication session;
• an interface and control layer for providing interactive controls for managing at least interaction with the virtual environment and the audiovisual content during the communication session;
• and
• a signaling server to initiate the communication session, negotiate parameters of the communication session, and relay control messages between the users of the communication session;
• wherein at least part of the audiovisual content of the communication layer is associated with the avatar of the local layer;
• wherein the at least the avatar and the audiovisual content is spatially associated with a position in the three dimensions within the virtual environment via metadata;
• and
• wherein the server renders the virtual environment using dynamic level of detail (LoD) techniques to conserve computational resources;
• wherein synchronization between the high-fidelity layer and the local layer are managed using timestamping to make corrective adjustments when discrepancies between the local layer and the high-fidelity layer are detected;
• wherein the signaling server further monitors quality of the communication session, provides feedback regarding the communication session to the users, and manages session resources relating to the communication session.

A method of providing a three-dimensional layered conferencing system with distributed computing for high-fidelity rendering of a virtual environment, the method being performed on a computerized device comprising a processor and memory with instructions being stored in the memory and operated from the memory to transform data, the method comprising:

• a) initiating a communication session via a signaling server;
• b) rendering remotely on a server a high-fidelity layer comprising the virtual environment in three dimensions;
• c) streaming the high-fidelity layer to a user device in approximately real-time over a network without intended delay for the communication session;
• d) rendering locally on the user device a local layer comprising an avatar for a user, which is separate and distinct from rendering of the virtual environment by the server, to be integrated into the virtual environment for the communication session, the local layer being synchronized with high-fidelity layer that is independently rendered on a server and streamed to the user device over the network;
• e) coordinating over a decentralized peer-to-peer network a communication layer comprising audiovisual content of the user communicated in approximately real-time without intended delay for the communication session;
• and
• f) providing interactive controls via an interface and control layer for managing at least interaction with the virtual environment and the audiovisual content during the communication session;
• wherein at least part of the audiovisual content of the communication layer is associated with the avatar of the local layer;
• wherein the at least the avatar and the audiovisual content is spatially associated with a position within the virtual environment in the three dimensions;
• and
• wherein the server renders the virtual environment using dynamic level of detail (LoD) techniques to conserve computational resources;
• wherein synchronization between the high-fidelity layer and the local layer are managed using timestamping to make corrective adjustments when discrepancies between the local layer and the high-fidelity layer are detected;
• wherein the signaling server additionally negotiates parameters of the communication session and relays control messages between the users of the communication session.