# US 11,907,717 B2 · Techniques for efficiently transferring data to a processor > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,907,717 B2 - Document ID: US11907717B2 - Status: Granted - Kind: B2 - Application number: 18/107,374 - Filed: 2023-02-08 - Published: 2023-06-15 - Granted: 2024-02-20 - Earliest claimed: 2019-10-29 - Assignees: NVIDIA Corporation - Applicants: NVIDIA Corporation - Inventors: Andrew Kerr; Jack Choquette; Xiaogang Qiu; Omkar Paranjape; Poornachandra Rao; Shirish Gadre; Steven J. Heinrich; Manan Patel; Olivier Giroux; Alan Kaatz - Web page: https://patentgrep.com/patent/US11907717B2 ## CPC - G06F 9/30043: {LOAD or STORE instructions; Clear instruction} - G06F 9/3009: {Thread control instructions} - G06F 9/3851: {from multiple instruction streams, e.g. multistreaming} - G06F 9/3885: {using a plurality of independent parallel functional units} - G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel} - G06F 9/522: {Barrier synchronisation} - G06F 12/0808: with cache invalidating means (G06F12/0815 takes precedence) - G06F 12/0888: using selective caching, e.g. bypass - G06F 9/3004: {to perform operations on memory} - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract A technique for block data transfer is disclosed that reduces data transfer and memory access overheads and significantly reduces multiprocessor activity and energy consumption. Threads executing on a multiprocessor needing data stored in global memory can request and store the needed data in on-chip shared memory, which can be accessed by the threads multiple times. The data can be loaded from global memory and stored in shared memory using an instruction which directs the data into the shared memory without storing the data in registers and/or cache memory of the multiprocessor during the data transfer. ## Claims ### Claim 1 A processing system comprising: a multithreaded processor configured to concurrently execute a plurality of threads; a plurality of data registers, each of the data registers assigned to executing threads; and on-chip memory including a cache memory and a shared memory, wherein the system is configured to, in response to at least one of the threads, execute a first instruction to retrieve data stored in a memory external to the processing system through the data registers and the cache memory and store the data into the shared memory, and in response to at least one of the threads, execute a second instruction to retrieve data stored in the memory external to the processing system and store the data into the shared memory without first storing the data in the data registers. ### Claim 2 (depends on 1) The processing system of claim 1, wherein the cache memory is configured to allow the plurality of executing threads to access tagged data stored in the cache memory, and the shared memory is configured to allow the plurality of executing threads to access untagged data stored in the shared memory. ### Claim 3 (depends on 2) The processing system of claim 2, wherein the shared memory is a software managed cache. ### Claim 4 (depends on 3) The processing system of claim 3, wherein the cache memory and the shared memory are a unified physical random access memory. ### Claim 5 (depends on 4) The processing system of claim 4, wherein the unified physical random access memory includes a register file including the plurality of data registers dynamically assignable to the executing threads. ### Claim 6 (depends on 1) The processing system of claim 1, wherein the processing system includes hardware circuitry configured to: receive a plurality of instructions to load data stored in the external memory and store the data into the shared memory, and determine, based on the plurality of instructions, a shared memory sector fill pattern for storing the retrieved data in the shared memory. ### Claim 7 (depends on 1) The processing system of claim 1, wherein the shared memory includes a plurality of sectors arranged in banks, and the retrieved data is stored into at least one of the sectors with an intra-sector swizzle. ### Claim 8 (depends on 1) The processing system of claim 1, wherein the shared memory includes a plurality of sectors arranged in banks, and portions of the retrieved data are stored into the sectors in different banks. ### Claim 9 (depends on 1) The processing system of claim 1, wherein the second instruction loads data from one sector of the external memory and stores the loaded data into parts of plural sectors in the shared memory. ### Claim 10 (depends on 1) The processing system of claim 1, wherein the retrieved data is stored in one or more cache lines of the cache memory and the system further comprises an interconnect circuit configured to transfer data stored in the one or more cache lines into sectors of the shared memory without first storing the retrieved data in the plurality of data registers. ### Claim 11 A processing system comprising: a multithreaded processor configured to concurrently execute a plurality of threads; a plurality of data registers, each of the data registers assigned to executing threads; and on-chip memory including a cache memory and a shared memory, wherein the system is configured to, in response to at least one of the threads, execute a first instruction to retrieve data stored in a memory external to the processing system through the data registers and the cache memory and store the data into the shared memory, and in response to at least one of the threads, execute a second instruction to retrieve data stored in the memory external to the processing system and store the data into the shared memory without first storing the data in the data registers and the cache memory. ### Claim 12 (depends on 11) The processing system of claim 11, wherein the cache memory is configured to allow the plurality of executing threads to access tagged data stored in the cache memory, and the shared memory is configured to allow the plurality of executing threads to access untagged data stored in the shared memory. ### Claim 13 (depends on 12) The processing system of claim 12, wherein the shared memory is a software managed cache. ### Claim 14 (depends on 12) The processing system of claim 12, wherein the cache memory and the shared memory are a unified physical random access memory. ### Claim 15 (depends on 14) The processing system of claim 14, wherein the unified physical random access memory includes a register file including the plurality of data registers dynamically assignable to the executing threads. ### Claim 16 A method performed by a processing system comprising: a multithreaded processor configured to concurrently execute a plurality of threads; a plurality of data registers, each of the data registers assigned to executing threads; and on-chip memory including a cache memory and a shared memory, the method comprising: at least one of the threads executing a first instruction to retrieve data stored in a memory external to the processing system through the data registers and the cache memory and store the data into the shared memory; and at least one of the threads executing a second instruction to retrieve data stored in the memory external to the processing system and store the data into the shared memory without first storing the data in the plurality of data registers. ### Claim 17 (depends on 16) The method of claim 16, wherein the cache memory is configured to allow the plurality of executing threads to access tagged data stored in the cache memory, and the shared memory is configured to allow the plurality of executing threads to access untagged data stored in the shared memory. ### Claim 18 (depends on 17) The method of claim 17, wherein the shared memory is a software managed cache. ### Claim 19 (depends on 17) The method of claim 17, wherein the cache memory and the shared memory are a unified physical random access memory. ### Claim 20 (depends on 19) The method of claim 19, wherein the unified physical random access memory includes a register file including the plurality of data registers dynamically assignable to the executing threads. ### Claim 21 (depends on 16) The method of claim 16, wherein the second instruction loads data from one sector of the external memory and stores the loaded data into parts of plural sectors in the shared memory.