Split rendering for extended reality (XR) applications

A method of operation by a client device, the method comprising:

• receiving composite video frames from a remote server node in an Edge Data Network (EDN);
• and
• for each composite video frame:
• extracting three-dimensional (3D) motion information from the composite video frame, wherein the composite video frame was generated based on the remote server node rendering a 3D scene containing one or more 3D objects to Z-order planes, with each 3D object assigned to a corresponding Z-order plane according to a Z-order distance of the 3D object, and creating a plurality of graphics layers, each graphics layer corresponding to a respective one of the Z-order planes, and each graphics layer having 3D motion information corresponding to each 3D object in the respective Z-order plane, and wherein different 3D objects in the respective Z-order plane having different 3D motion information belong to respective graphics layer groups of the corresponding graphics layer;
• decoding the composite video frame into the plurality of graphics layers, including separately decoding each graphics layer group of each graphics layer;
• determining a latency time based on a time stamp of the composite video frame and a current rendering time;
• correcting positions of the respective graphics layers based on the latency time and the 3D motion information corresponding to the one or more 3D objects, and wherein the correcting accounts for movement in the image plane and in a Z-order direction;
• and
• rendering the corrected plurality of graphics layers on a display to the user.

A server node in an Edge Data Network (EDN), the server node comprising:

• communications interface circuitry configured to communicate with a client device operatively connected to the EDN;
• and
• processing circuitry configured to:
• render the 3D scene to Z-order planes, with each 3D object assigned to a corresponding Z-order plane according to a Z-order distance of the 3D object;
• create a plurality of graphics layers, each graphics layer corresponding to a respective one of the Z-order planes, and each graphics layer having 3D motion information for each 3D object in the respective Z-order plane, and wherein different 3D objects in the respective Z-order plane having different 3D motion information are assigned to respective graphics layer groups of the corresponding graphics layer;
• encode the graphics layer groups separately into a composite video frame such that each graphics layer group is separately decodable and the composite video frame includes the 3D motion information corresponding to each graphics layer group;
• and
• send the composite video frame to the client device for decoding, according to the graphics layer groupings and the 3D motion information corresponding to each of the one or more 3D objects.

A client device comprising:

• communications interface circuitry configured to communicate with a server node in an Edge Data Network (EDN);
• and
• processing circuitry configured to:
• receive composite video frames from the server node;
• and
• for each composite video frame:
• extract three-dimensional (3D) motion information from the composite video frame, wherein the composite video frame was generated based on the remote server node rendering a 3D scene containing one or more 3D objects to Z-order planes, with each 3D object assigned to a corresponding Z-order plane according to a Z-order distance of the 3D object, and creating a plurality of graphics layers, each graphics layer corresponding to a respective one of the Z-order planes, and each graphics layer having 3D motion information corresponding to each 3D object in the respective Z-order plane, and wherein different 3D objects in the respective Z-order plane having different 3D motion information belong to respective graphics layer groups of the corresponding graphics layer;
• decode the composite video frame into the plurality of graphics layers, including separately decoding each graphics layer group of each graphics layer:
• determine a latency time based on a time stamp of the composite video frame and a current rendering time;
• correct positions of the respective graphics layers based on the latency time and the 3D motion information corresponding to the one or more 3D objects, and wherein the correcting accounts for movement in the image plane and in a Z-order direction;
• and
• render the corrected plurality of graphics layers on a display to the user.