Memory prefetching in multiple GPU environment

One or more non-transitory computer-readable storage mediums having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• generating a prefetch instruction for one or more prefetch operations by a prefetcher of a first graphics processing unit (GPU), the first GPU being one GPU of a plurality of GPUs in a computing system, the prefetch instruction being directed to a memory including a plurality of memory elements, each of the plurality of GPUs being coupled with one or more of the plurality of memory units, the plurality of memory units providing a unified virtual memory that is accessible to each of the plurality of GPUs, the unified virtual memory including at least a first page and a second page, the first page and the second page being adjacent in the unified virtual memory;
• and
• caching prefetched data in a cache of the first GPU;
• wherein, in a prefetch operation by the first GPU that includes a prefetch to the first page:
• the prefetcher of the first GPU is allowed to prefetch data from the first page if it is owned by the first GPU or a host processor of the computing system, and is prohibited from prefetching from the first page if the first page is owned by a GPU other than the first GPU, and
• upon allowing the prefetch from the first page, the prefetch of data by the prefetcher of the first GPU is halted upon reaching a page boundary between the first page and the second page or upon reaching a boundary of a memory surface;
• and
• wherein the prefetching of data by the plurality of GPUs does not recognize a physical structure of the memory in the unified virtual memory.

A method comprising:

• generating a prefetch instruction for one or more prefetch operations by a prefetcher of a first graphics processing unit (GPU), the first GPU being one GPU of a plurality of GPUs in a computing system, the prefetch instruction being directed to a memory including a plurality of memory elements, each of the plurality of GPUs being coupled with one or more of the plurality of memory units, the plurality of memory units providing a unified virtual memory that is accessible to each of the plurality of GPUs, the unified virtual memory including at least a first page and a second page, the first page and the second page being adjacent in the unified virtual memory;
• and
• caching prefetched data in a cache of the first GPU;
• wherein, in a prefetch operation by the first GPU that includes a prefetch to the first page:
• the prefetcher of the first GPU is allowed to prefetch data from the first page if it is owned by the first GPU or a host processor of the computing system, and is prohibited from prefetching from the first page if the first page is owned by a GPU other than the first GPU, and
• upon allowing the prefetch from the first page, the prefetch of data by the prefetcher of the first GPU is halted upon reaching a page boundary between the first page and the second page or upon reaching a boundary of a memory surface;
• and
• wherein the prefetching of data by the plurality of GPUs does not recognize a physical structure of the memory in the unified virtual memory.