Artificially rendering images using viewpoint interpolation and extrapolation

A system comprising:

• a processor;
• and
• memory, the memory storing instructions to execute a method, the method comprising:
• obtaining a content model from a surround view, wherein the surround view is a multi-view interactive digital media representation and the content model is a foreground three-dimensional model including at least one object, wherein the content model comprises one or more layers;
• obtaining a context model from the surround view, wherein the context model is a background panoramic representation including scenery surrounding the at least one object, wherein the context model comprises one or more layers;
• and
• generating an artificially rendered image corresponding to a first viewpoint by projecting the content model into a first frame corresponding to the first viewpoint and projecting the panoramic representation into the first frame, wherein the artificially rendered image is generated by moving different layers in different motions, wherein generating the artificially rendered image includes generating using viewpoint-interpolation and any missing information is filled in using viewpoint-extrapolation, wherein viewpoint-extrapolation is achieved by moving different layers in a motion perpendicular to the trajectory from the first frame to a second frame.

A non-transitory computer readable medium storing instructions to execute a method, the method comprising:

• obtaining a content model from a surround view, wherein the surround view is a multi-view interactive digital media representation and the content model is a foreground three-dimensional model including at least one object, wherein the content model comprises one or more layers;
• obtaining a context model from the surround view, wherein the context model is a background panoramic representation including scenery surrounding the at least one object, wherein the context model comprises one or more layers;
• and
• generating an artificially rendered image corresponding to a first viewpoint by projecting the content model into a first frame corresponding to the first viewpoint and projecting the panoramic representation into the first frame, wherein the artificially rendered image is generated by moving different layers in different motions, wherein generating the artificially rendered image includes generating using viewpoint-interpolation and any missing information is filled in using viewpoint-extrapolation, wherein viewpoint-extrapolation is achieved by moving different layers in a motion perpendicular to the trajectory from the first frame to a second frame.