Systems and methods for network-based media processing

A network-based media processing (NBMP) system, comprising:

• a media control plane implementing a first network protocol, and including a media origin server (MOS);
• a network control plane implementing a second network protocol, and including a network platform orchestrator (NPO), a first network orchestration client (NOC), a network resource element (NRE), and a plurality of media processing elements (MPEs), wherein the NPO is configured for operable communication with (i) the MOS through the media control plane, (ii) the first NRE through the network control plane, and (iii) a first client device through the network control plane;
• a media data plane implementing a third network protocol, and including a first media resource within the NRE;
• a mixed content media interchange format (MIF) configured to provide a unified data format for delivering mixed content media having both natural content and synthetic content, according to a format requested from the first client device and a second client device different than the first client device;
• and
• a first hybrid natural/synthetic scene (HNSS) container configured to transform raw raster natural content into geometric format data,
• wherein the first network protocol is different from the second network protocol,
• wherein the NRE is configured for operable communication with (i) the first display unit of the first client device through a second 1MPE of the plurality of MPEs, and (ii) the second display unit of the second client device through a third MPE of the plurality of MPEs,
• wherein the first client device includes sufficient user resources to process the mixed content media, wherein the second client device does not include sufficient user resources to process the mixed content media, and wherein the mixed content MIF is further configured to format the mixed content media to enable the NRE to deliver the mixed content media (i) to the first client device according to a first resource capability of the first display unit, and (ii) to the second client device according to a second resource capability of the second display unit,
• wherein the NRE is further configured to deliver the mixed content media to the first display unit and the second display unit in at least one of an augmented reality format and an augmented virtuality format, and
• wherein the raw raster natural content includes data capture of at least one of surface, texture, and reflectance information of a scene, and wherein the geometric format data includes at least one of (i) points in three dimensional space, (ii) texture meshes, and (iii) layers of mapping information.

A network-based media processor configured for operable communication with a content provider, a remote first client device, and a remote second client device different from the remote first client device, the network-based media processor comprising:

• a hybrid natural/synthetic scene (HNSS) container configured to transform raw raster natural content from the content provider into geometric format data;
• and
• a mixed content media interchange format (MIF) configured to provide a unified data format for (i) receiving input mixed reality media content having a mixture of both natural content and synthetic content, and (ii) outputting HNSS content to the remote first client device according to a first target format corresponding to first user resources of the remote first client device, and (iii) outputting the HNSS content to the remote second client device according to a second target format corresponding to second user resources of the remote second client device,
• wherein the network-based media processor is configured to deliver the mixed reality content media to the remote first client device and to the remote second client device in at least one of an augmented reality format and an augmented virtuality format,
• wherein the raw raster natural content includes data capture of at least one of surface, texture, and reflectance information of a scene, and wherein the geometric format data includes at least one of (i) points in three dimensional space, (ii) texture meshes, and (iii) layers of mapping information,
• wherein the first user resources are greater than second the user resources, and
• wherein the second user resources are insufficient to independently process the mixed reality content media apart from the second target format.