System and method enabling interactions in virtual environments with virtual presence

A system enabling spatial video-based virtual presence, comprising:

• one or more server computers comprising at least one processor and memory storing data and instructions implementing a virtual environment platform comprising at least one virtual environment;
• at least one sending client device comprising at least one processor and memory storing data and instructions configured to, when executed by the at least one processor, cause the at least one sending client device to:
• generate a user real-time 3D virtual cutout by obtaining a 2D or 3D live video data feed from at least one camera and performing a background removal thereof;
• and
• transmit the user real-time 3D virtual cutout for presentation by at least one receiving client device;
• and
• at least one receiving client device comprising at least one processor and memory storing data and instructions configured to, when executed by the at least one processor, cause the at least one receiving client device to:
• receive the user real-time 3D virtual cutout;
• and
• insert and combine the user real-time 3D virtual cutout with the virtual environment;
• wherein the virtual environment is shared by the at least one sending client device and the at least one receiving client device and corresponding user real-time 3D virtual cutouts, enabling multi-user shared experiences with spatial video-based virtual presence within the virtual environment;
• and
• wherein an orientation of the user real-time 3D virtual cutout is updated automatically by tracking and analyzing one or more of user eye-and-head-tilting data and head-rotation data using the 2D or 3D live video data feed from the at least one camera.