Media compositor for computer-generated reality

A system comprising:

• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
• obtaining a first data stream comprising rendered frames, the rendered frames comprising rendered frame content representing real content and virtual content rendered during a user experience at a plurality of instants in time for at least two different viewpoints;
• obtaining a second data stream comprising additional data relating to the user experience at the plurality of instants in time and based on the at least two different viewpoints;
• and
• forming a composited stream based on the first data stream and the second data stream for the at least two different viewpoints rendered during the user experience, wherein the composited stream:
• comprises composited frames that each comprise a time-stamped n-dimensional image corresponding to a single instant in time,
• aligns the rendered frame content with the additional data to record content for the plurality of instants in time, and
• comprises another viewpoint of the recorded content for a viewer, wherein the another viewpoint is a different viewpoint than the at least two different viewpoints rendered during the user experience.

A non-transitory computer-readable storage medium, storing program instructions computer-executable on a computer to perform operations comprising:

• obtaining a first data stream comprising rendered frames, the rendered frames comprising rendered frame content representing real content and virtual content rendered during a user experience at a plurality of instants in time for at least two different viewpoints;
• obtaining a second data stream comprising additional data relating to the user experience at the plurality of instants in time and based on the at least two different viewpoints;
• and
• forming a composited stream based on the first data stream and the second data stream for the at least two different viewpoints rendered during the user experience, wherein the composited stream:
• comprises composited frames that each comprise a time-stamped n-dimensional image corresponding to a single instant in time,
• aligns the rendered frame content with the additional data to record content for the plurality of instants in time, and
• comprises another viewpoint of the recorded content for a viewer, wherein the another viewpoint is a different viewpoint than the at least two different viewpoints rendered during the user experience.