No miss cache structure for real-time image transformations with data compression
AssigneeMicrosoft Technology Licensing, LLC
InventorsTolga Ozguner, Gene Leung, Jeffrey Powers Bradford, Adam James Muff, Miguel Comparan, Ryan Scott Haraden, Christopher Jon Johnson
Life of this patent
Estimating the term…
Abstract
Systems and methods are disclosed herein for providing improved cache structures and methods that are optimally sized to support a predetermined range of late stage adjustments and in which image data is intelligently read out of DRAM and cached in such a way as to eliminate re-fetching of input image data from DRAM and minimize DRAM bandwidth and power. The systems and methods can also be adapted to work with compressed image data.
Claims24
An improved method for caching image data intended for use with a head-mounted, augmented, mixed or virtual reality display system (HMD) having a graphics processing unit (GPU), a holographic processing unit (HPU) with an on-chip cache, system memory, and a display, the GPU generating image that can include text, holographic objects and other visual elements that can be displayed on the display, the system memory for storing GPU images as compressed image data, and the HPU configured to perform late stage adjustments to correct GPU images for movement of the HMD that may occur during the image rendering process, and the cache for constructing an output image based on the GPU generated image and the late stage adjustments, and the HPU generating a late stage adjustment matrix that maps each pixel of the late stage adjusted output image to the corresponding pixel of the GPU image, the method comprising acts for:
- pre-fetching and decompressing one or more tiles of compressed GPU image data from system memory and placing the decompressed GPU image data in a first cache, the one or more tiles being selected in the order needed to construct the output image;
- pre-fetching one or more rows of decompressed image data from the first cache, the one or more rows being selected in the order needed to construct the output image;
- writing the pre-fetched one or more rows of decompressed GPU image data into a second cache in the order needed to construct the output image based on the late stage adjustment matrix;
- and
- constructing the output image by walking the one or more rows of the GPU image in the second cache according to the adjustment matrix and outputting the output image.
An improved method for caching image data intended for use with a head-mounted, augmented, mixed or virtual reality display system (HMD) having a graphics processing unit (GPU), a holographic processing unit (HPU) with an on-chip cache, system memory, and a display, the GPU generating image that can include text, holographic objects and other visual elements that can be displayed on the display, the system memory for storing GPU images as compressed image data, and the HPU configured to perform late stage adjustments to correct GPU images for movement of the HMD that may occur during the image rendering process, and the cache for constructing an output image based on the GPU generated image and the late stage adjustments, and the HPU generating a late stage adjustment matrix that maps each pixel of the late stage adjusted output image to the corresponding pixel of the GPU image, the method comprising acts for:
- pre-fetching one or more tiles of compressed GPU image data from system memory and placing the compressed GPU image data in a first cache, the one or more tiles of compressed GPU image data being selected in the order needed to construct the output image;
- pre-fetching and decompressing one or more tiles of compressed GPU image data from the first cache and placing the decompressed GPU image data in a second cache, the one or more tiles being selected in the order needed to construct the output image;
- pre-fetching one or more rows of decompressed GPU image data from the second cache and writing the pre-fetched one or more rows of decompressed GPU image data into a third cache in the order needed to construct the output image based on the late stage adjustment matrix;
- and
- constructing the output image by walking the one or more rows of the decompressed GPU image data in the third cache according to the adjustment matrix and outputting the output image.
Dependencies are inferred from the claim text.