Systems and methods for immersive digital experiences

A computer system comprising:

• at least one memory;
• and at least one processor coupled to the at least one memory, the system configured to cause the at least one processor to execute instructions stored in the at least one memory to:
• generate, a virtual environment, the virtual environment including positional data for at least one digital object;
• receive, from a first device, first streamed motion data, the first streamed motion data corresponding to real-time motion of a first physical subject and including a first plurality of data points, each data point of the first plurality of data points being associated with a position of a portion of the first physical subject and including at least a portion of a unit quaternion;
• receive, from a second device, second streamed motion data, the second streamed motion data corresponding to real-time motion of a second physical subject and including a second plurality of data points, each data point of the second plurality of data points being associated with a position of a portion of the second physical subject;
• generate, within the virtual environment, a first skeletal mesh and a second skeletal mesh;
• position the first skeletal mesh at a first initial position relative to the at least one digital object;
• position the second skeletal mesh at a second initial position relative to the at least one digital object;
• associate each of the first plurality of data points with a corresponding point along the first skeletal mesh;
• associate each of the second plurality of data points with a corresponding point along the second skeletal mesh;
• and
• stream, in real time, to a client device, positional information of the first skeletal mesh, the second skeletal mesh, and the at least one digital object.

A non-transitory computer readable medium, the non-transitory computer readable medium comprising instructions to cause at least one processor on a computer to:

• generate, a virtual environment, the virtual environment including positional data for at least one digital object;
• receive, from a first device, first streamed motion data, the first streamed motion data corresponding to real-time motion of a first physical subject and including a first plurality of data points, each data point of the first plurality of data points being associated with a position of a portion of the first physical subject, wherein the first streamed motion data is compressed;
• decompressing the first streamed motion data by inferring from the first plurality of data points, an inferred data point, a position of the inferred data point being calculated relative to at least one data point of the first plurality of data points;
• receive, from a second device, second streamed motion data, the second streamed motion data corresponding to real-time motion of a second physical subject and including a second plurality of data points, each data point of the second plurality of data points being associated with a position of a portion of the second physical subject;
• generate, within the virtual environment, a first skeletal mesh and a second skeletal mesh;
• position the first skeletal mesh at a first initial position relative to the at least one digital object;
• position the second skeletal mesh at a second initial position relative to the at least one digital object;
• associate each of the first plurality of data points with a corresponding point along the first skeletal mesh;
• associate each of the second plurality of data points with a corresponding point along the second skeletal mesh;
• and
• stream, in real time, to a client device, positional information of the first skeletal mesh, the second skeletal mesh, and the at least one digital object.

A method for providing an immersive digital experience, the method comprising:

• generating, at a first computing device, a virtual environment, the virtual environment including positional data for at least one digital object;
• receiving first streamed motion data, the first streamed motion data corresponding to real-time motion of a first physical subject, wherein the first streamed motion data includes a first positional coordinate and a first plurality of data points, each data point of the first plurality of data points being associated with a position of a portion of the first physical subject;
• generating, within the virtual environment, a first skeletal mesh and positioning the first skeletal mesh at a first initial position relative to the at least one digital object;
• associating each of the first plurality of data points with a corresponding point along the skeletal mesh;
• determining that the first positional coordinate does not correspond to a position of a corresponding portion of the first physical subject, wherein determining that the first positional coordinate does not correspond to a position of a corresponding portion of the first subject includes determining that the first positional coordinate is located outside of a first range of possible positions of the corresponding portion of the first subject;
• generating an updated first positional coordinate corresponding to a predicted position of the corresponding portion of the first physical subject;
• and
• associating the updated first positional coordinate with a corresponding point along the skeletal mesh;
• and
• streaming, to a client device, positional information of the first skeletal mesh and the at least one digital object.

A method for providing an immersive digital experience, the method comprising:

• generating, at a first computing device, a virtual environment, the virtual environment including positional data for at least one digital object;
• receiving first streamed motion data, the first streamed motion data corresponding to real-time motion of a first physical subject, wherein the first streamed motion data includes a first positional coordinate and a first plurality of data points, each data point of the first plurality of data points being associated with a position of a portion of the first physical subject;
• generating, within the virtual environment, a first skeletal mesh and positioning the first skeletal mesh at a first initial position relative to the at least one digital object;
• associating each of the first plurality of data points with a corresponding point along the skeletal mesh;
• determining that the first positional coordinate does not correspond to a position of a corresponding portion of the first physical subject;
• generating an updated first positional coordinate corresponding to a predicted position of the corresponding portion of the first physical subject, the first positional coordinate being generated using an inverse kinematics algorithm;
• and
• associating the updated first positional coordinate with a corresponding point along the skeletal mesh;
• and
• streaming, to a client device, positional information of the first skeletal mesh and the at least one digital object.

A method for providing an immersive digital experience, the method comprising:

• generating, at a first computing device, a virtual environment, the virtual environment including positional data for at least one digital object;
• receiving first streamed motion data, the first streamed motion data corresponding to real-time motion of a first physical subject and including a first plurality of data points, each data point of the first plurality of data points being associated with a position of a portion of the first physical subject and including at least a portion of a unit quaternion;
• generating, within the virtual environment, a first skeletal mesh and positioning the first skeletal mesh at a first initial position relative to the at least one digital object;
• associating each of the first plurality of data points with a corresponding point along the skeletal mesh;
• and
• streaming, to a client device, positional information of the first skeletal mesh and the at least one digital object.