Efficient Motion-Compensated Spatiotemporal Sampling

One or more computer-readable non-transitory storage media embodying software that is operable when executed to:

• access a video comprising a first frame and a second frame;
• determine first sampling locations for the first frame;
• determine second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame;
• select a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations;
• define one or more rejection areas in the second frame based on the subset of the second sampling locations;
• determine third sampling locations for the second frame in areas outside of the one or more rejection areas;
• and
• generate a sample of the video based on the pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the third sampling locations.

A system comprising:

• one or more processors;
• and
• one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:
• access a video comprising a first frame and a second frame;
• determine first sampling locations for the first frame;
• determine second sampling locations for the second frame by transforming the first sampling locations to the second frame according to an optical flow between the first frame and the second frame;
• select a subset of the second sampling locations based on a comparison between pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the second sampling locations;
• define one or more rejection areas in the second frame based on the subset of the second sampling locations;
• determine third sampling locations for the second frame in areas outside of the one or more rejection areas;
• and
• generate a sample of the video based on the pixels in the first frame corresponding to the first sampling locations and pixels in the second frame corresponding to the third sampling locations.