Generation, transmission and rendering of virtual reality multimedia

A generation computing device, comprising:

• a memory;
• a network interface;
• and
• a processor interconnected with the memory and the network interface, the processor configured to:
• obtain point cloud data at a processor of a generation computing device, the point cloud data including colour and three-dimensional position data for each of a plurality of points corresponding to locations in a capture volume;
• generate primary image data containing (i) a first projection of a first subset of the points into a two-dimensional frame of reference, and (ii) for each point of the first subset, depth data derived from the corresponding position data;
• generate secondary image data containing (i) a second projection of a second subset of the points into the two-dimensional frame of reference, the second projection overlapping with at least part of the first projection in the two-dimensional frame of reference, and (ii) for each point of the second subset, depth data derived from the corresponding position data;
• and
• store the primary image data and the secondary image data in the memory.

A method of rendering virtual reality multimedia data, comprising:

• obtaining primary image data containing (i) a first projection of a first subset of points in a three-dimensional point cloud into a two-dimensional frame of reference, and (ii) for each point of the first subset, depth data derived from corresponding position data of the points;
• obtaining secondary image data containing (i) a second projection of a second subset of the points into the two-dimensional frame of reference, the second projection overlapping with at least part of the first projection in the two-dimensional frame of reference, and (ii) for each point of the second subset, depth data derived from the corresponding position data;
• receiving a viewpoint position from a virtual reality display;
• selecting at least a portion of the primary image data and the secondary image data based on the viewpoint position;
• and
• rendering the selected primary and secondary image data on the virtual reality display.

A client computing device, comprising:

• a memory;
• a network interface;
• and
• a processor interconnected with the memory and the network interface, the processor configured to:
• obtain primary image data containing (i) a first projection of a first subset of points in a three-dimensional point cloud into a two-dimensional frame of reference, and (ii) for each point of the first subset, depth data derived from corresponding position data of the points;
• obtain secondary image data containing (i) a second projection of a second subset of the points into the two-dimensional frame of reference, the second projection overlapping with at least part of the first projection in the two-dimensional frame of reference, and (ii) for each point of the second subset, depth data derived from the corresponding position data;
• receive a viewpoint position from a virtual reality display;
• select at least a portion of the primary image data and the secondary image data based on the viewpoint position;
• and
• render the selected primary and secondary image data on the virtual reality display.