Three-dimensional motion mapping for cloud gaming

A system for implementing three-dimensional motion mapping, comprising:

• a processor;
• a memory;
• and
• computer-coded instructions embodied in the memory and executable by the processor, wherein the computer coded instructions are configured to implement a method for three-dimensional motion mapping of a current frame of a video game, the method comprising:
• a) performing a reverse three-dimensional transform for a set of anchor pixels in a current frame of video, thereby producing a set of transformed anchor pixels by mapping two-dimensional coordinates associated with a pixel in the current frame to three-dimensional object coordinates, transforming the three-dimensional object coordinates into three-dimensional object coordinates associated with a corresponding pixel in a previous frame, and mapping the three-dimensional object coordinates associated with the corresponding pixel in the previous frame to two-dimensional coordinates associated with the corresponding pixel in a previous frame;
• b) for each reverse transformed anchor pixel, indicating whether the reverse transformed anchor pixel is occluded or visible, whereby the reverse transformed anchor pixel is indicated as occluded if an estimated depth for a corresponding pixel in a previous frame is greater than a real depth for the corresponding pixel in the previous frame, and whereby the reverse transformed anchor pixel is indicated as not occluded and therefore visible if the estimated depth for the corresponding pixel in the previous frame is not greater than a real depth for the corresponding pixel in the previous frame;
• c) constructing a motion vector map for at least a subset of the set of reverse transformed anchor pixels;
• d) deriving a best target motion vector for each section in a first set of sections in the current frame using the motion vector map or performing motion estimation for each section in a second set of sections in the current frame using the motion vector map, wherein the sections in the second set include occluded pixels;
• e) deriving target motion vectors for one or more groups of two or more sections of the current frame, each group comprising two or more adjoining sections each with best target motion vectors;
• and
• f) encoding the current frame using the target motion vectors, the best target motion vector for each section, or motion estimation results.

A computer program product comprising:

• a non-transitory, computer-readable storage medium having computer readable program code embodied in said medium for implementing a method for three-dimensional motion mapping of a current video frame in a computer game, the method comprising:
• a) performing a reverse three-dimensional transform for a set of anchor pixels in a current frame of video, thereby producing a set of transformed anchor pixels by mapping two-dimensional coordinates associated with a pixel in the current frame to three-dimensional object coordinates, transforming the three-dimensional object coordinates into three-dimensional object coordinates associated with a corresponding pixel in a previous frame, and mapping the three-dimensional object coordinates associated with the corresponding pixel in the previous frame to two-dimensional coordinates associated with the corresponding pixel in a previous frame;
• b) for each reverse transformed anchor pixel, indicating whether the reverse transformed anchor pixel is occluded or visible, whereby the reverse transformed anchor pixel is indicated as occluded if an estimated depth for a corresponding pixel in a previous frame is greater than a real depth for the corresponding pixel in the previous frame, and whereby the reverse transformed anchor pixel is indicated as not occluded and therefore visible if the estimated depth for the corresponding pixel in the previous frame is not greater than a real depth for the corresponding pixel in the previous frame;
• c) constructing a motion vector map for at least a subset of the set of reverse transformed anchor pixels;
• d) deriving a best target motion vector for each section in a first set of sections in the current frame using the motion vector map or performing motion estimation for each section in a second set of sections in the current frame using the motion vector map, wherein the sections in the second set include occluded pixels;
• e) deriving target motion vectors for one or more groups of two or more sections of the current frame, each group comprising two or more adjoining sections each with best target motion vectors;
• and
• f) encoding the current frame using the target motion vectors, the best target motion vector for each section, or motion estimation results.