Depth-aware reprojection

On a computing system comprising a display and a sensor subsystem, a method comprising:

• receiving image data to be displayed via the display;
• receiving a depth map for the image data;
• storing the image data in a frame buffer;
• processing the depth map to obtain a blurred depth map;
• based upon motion data received from the sensor subsystem, determining a translation to be made to the image data prior to display;
• and
• for each pixel of a plurality of pixels in the frame buffer, based upon the translation determined,
• translating an original ray extending from an original virtual camera location to a corresponding original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location,
• determining a location at which the reprojected ray intersects the blurred depth map, and
• sampling a color of a pixel in the frame buffer for display at the reprojected frame buffer location based upon a color in the frame buffer corresponding to the location at which the reprojected ray intersects the blurred depth map.

A head-mounted display device comprising a display, a sensor subsystem, and a reprojection subsystem configured to:

• receive image data to be displayed via the display;
• receive depth data for the image data;
• store the image data in a frame buffer;
• process the depth map to obtain a blurred depth map;
• based upon motion data received from the sensor subsystem, determine a translation to be made to the image data prior to display;
• and
• for each pixel of a plurality of pixels in the frame buffer, based upon the translation determined,
• translate an original ray extending from an original virtual camera location to a corresponding original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location,
• determine a location at which the reprojected ray intersects the blurred depth map using successive approximation, and
• sample a color of a pixel in the frame buffer for display at the reprojected frame buffer location based upon a color in the frame buffer corresponding to the location at which the reprojected ray intersects the blurred depth map.