Foveated image rendering for head-mounted display devices

One or more non-transitory computer readable storage media comprising computer readable instructions which, when executed, cause one or more processors of a head-mounted display device to at least:

• store first image data with a first resolution from a host device at the head-mounted display device;
• obtain second image data from the host device via a data interface after the first image data is stored at the head-mounted display device, the second image data to have a second resolution lower than the first resolution;
• as the second image data is obtained from the host device via the data interface, up-sample the second image data based on first metadata to generate up-sampled second image data to combine with the first image data, the up-sampled second image data to have the first resolution;
• combine the stored first image data and the up-sampled second image data based on second metadata from the host device to form a foveated image frame;
• and
• apply, based on third metadata from the host device, a combination of at least two different filter operations on a rectangular strip of the foveated image frame that corresponds to an overlap region including a portion of the first image data and a portion of the up-sampled second image data to render the foveated image frame on the display of the head-mounted display device, the third metadata to specify a width in pixels of the rectangular strip.

An apparatus to implement foveated image rendering, the apparatus comprising:

• means for storing first image data for a foveated image frame from a host device, the first image data to have a first resolution, the means for storing to store the first image data from the host device before second image data for the foveated image frame is obtained from the host device;
• means for up-sampling the second image data based on first metadata from the host device to generate up-sampled second image data to combine with the first image data, the means for up-sampling to up-sample the second image data as the second image data is obtained from the host device, the second image data to have a second resolution lower than the first resolution, the up-sampled second image data to have the first resolution;
• means for combining the first image data from the means for storing and the up-sampled second image data based on second metadata from the host device to form the foveated image frame;
• and
• means for filtering the foveated image frame, the means for filtering to apply, based on third metadata from the host device, a combination of at least two different filter operations on a rectangular strip of the foveated image frame that corresponds to an overlap region including a portion of the first image data and a portion of the up-sampled second image data to render the foveated image frame on a display of a head-mounted display device, the third metadata to specify a width in pixels of the rectangular strip.

A system to implement foveated image rendering, the system comprising:

• a data interface;
• a host device to generate downsized image data associated with a low-resolution region of a foveated image frame;
• and
• a head-mounted display device in communication with the host device via the data interface, the head-mounted display device including a display with multiple screens, the head-mounted display device to:
• as the downsized image data is obtained from the host device via the data interface, up-sample, based on first metadata from the host device, the downsized image data associated with the low-resolution region of the foveated image frame to generate up-sampled image data associated with the low-resolution region of the foveated image frame;
• combine, based on second metadata from the host device, the up-sampled image data associated with the low-resolution region with previously stored image data associated with a high-resolution region of the foveated image frame to form the foveated image frame;
• and
• apply, based on third metadata from the host device, a combination of at least two different filter operations on a rectangular strip of the foveated image frame that corresponds to an overlap region including a portion of the high-resolution region and a portion of the low-resolution region to render the foveated image frame, the third metadata to specify a width in pixels of the rectangular strip.