Maximized memory throughput using cooperative thread arrays

A method for processing an input data stream comprising a plurality of input data elements, the method comprising:

• storing the input data elements of the input data stream in memory;
• defining a number of thread arrays to be executed concurrently by parallel processing hardware, each thread array comprising a number of concurrent threads, each thread having a unique thread identifier and each thread array having a unique array identifier, wherein each thread is assigned to process one or more of the input data elements, an input data element for a given thread being selected from an address in memory that is a function of the unique thread identifier and the unique array identifier associated with the thread;
• executing, using the parallel processing hardware, the number of thread arrays to process the input data stream and write an output data stream to the memory, wherein executing one of the plurality of thread arrays includes:
• organizing the threads of the thread array into a plurality of SIMD groups, wherein at least a first one of the SIMD groups includes a plurality of threads;
• and
• retrieving the input data elements for all threads of the first SIMD group from the memory in a single memory access operation,
• wherein executing one of the plurality of thread arrays further includes:
• determining, for each thread of each SIMD group, a second data element of the input data stream to access, wherein the second data element for each thread of each SIMD group is separated from the first data element by a stride that is the same for all of the threads of all of the SIMD groups.