System for and method of generating user-selectable novel views on a viewing device

A server system comprising:

• a central processing unit and a non-transitory computer-readable memory that stores instructions, when executed by the central processing unit, causing the server system to perform:
• joining a plurality of foreground models using image data from a plurality of cameras into a foreground model;
• creating foreground mapping data for foreground image data in a visual atlas to a first set of three-dimensional (3D) coordinates in a 3D data representation;
• projecting environment image data of the plurality of cameras for each camera onto respective environment model;
• creating environment mapping data for the environment image data in the respective environment model to a second set of 3D coordinates in the 3D data representation;
• saving the respective environment model for each camera as an environment image;
• and
• providing data based on the foreground model and the environment images to a viewer system, wherein the viewer system is to project foreground data on the environment images.