Large scene neural view synthesis

A method performed by one or more computers, the method comprising:

• maintaining a plurality of view synthesis models, wherein each view synthesis model corresponds to a respective sub-region of a scene of an environment and is configured to receive an input specifying a camera viewpoint in the corresponding sub-region and to generate as output a synthesized image of a scene from the camera viewpoint and wherein each view synthesis model has been independently trained on a respective set of training data that includes images captured from viewpoints within the corresponding sub-region of the scene;
• obtaining an input specifying a new camera viewpoint;
• for each of a subset of the plurality of view synthesis models, processing a respective input specifying the new camera viewpoint to generate, as output, data characterizing a synthesized image of the scene from the new camera viewpoint;
• and
• combining the data characterizing the synthesized images generated by the view synthesis models in the subset to generate a final synthesized image of the scene from the new camera viewpoint.

A system comprising one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the one more computers to perform operations comprising:

• maintaining a plurality of view synthesis models, wherein each view synthesis model corresponds to a respective sub-region of a scene of an environment and is configured to receive an input specifying a camera viewpoint in the corresponding sub-region and to generate as output a synthesized image of a scene from the camera viewpoint and wherein each view synthesis model has been independently trained on a respective set of training data that includes images captured from viewpoints within the corresponding sub-region of the scene;
• obtaining an input specifying a new camera viewpoint;
• for each of a subset of the plurality of view synthesis models, processing a respective input specifying the new camera viewpoint to generate, as output, data characterizing a synthesized image of the scene from the new camera viewpoint;
• and
• combining the data characterizing the synthesized images generated by the view synthesis models in the subset to generate a final synthesized image of the scene from the new camera viewpoint.

One or more non-transitory computer-readable storage media storing instructions that when executed by one or more computers cause the one more computers to perform operations comprising

• maintaining a plurality of view synthesis models, wherein each view synthesis model corresponds to a respective sub-region of a scene of an environment and is configured to receive an input specifying a camera viewpoint in the corresponding sub-region and to generate as output a synthesized image of a scene from the camera viewpoint and wherein each view synthesis model has been independently trained on a respective set of training data that includes images captured from viewpoints within the corresponding sub-region of the scene;
• obtaining an input specifying a new camera viewpoint;
• for each of a subset of the plurality of view synthesis models, processing a respective input specifying the new camera viewpoint to generate, as output, data characterizing a synthesized image of the scene from the new camera viewpoint;
• and
• combining the data characterizing the synthesized images generated by the view synthesis models in the subset to generate a final synthesized image of the scene from the new camera viewpoint.