Efficient work unit processing in a multicore system

A device comprising:

• a plurality of processing units, each of the processing units configured to execute one or more of a plurality of work unit handlers (WU handlers) for processing stream fragments, and wherein each of the processing units include a level one (L1) buffer cache;
• a memory to store the stream fragments;
• a plurality of queues configured to hold work units, each of the work units associated with one or more stream fragments, and wherein each of the work units identifies one of the WU handlers for processing the one or more stream fragments;
• and
• a load store unit configured to:
• identify, prior to completion of processing of a first work unit by a first processing unit of the plurality of processing units, a second work unit that is expected to be processed by the first processing unit after the first work unit, wherein the first processing unit processes the first work unit by accessing first work unit data associated with the first work unit in an active segment of the L1 buffer cache included within the first processing unit and generating, from the first work unit data, modified first work unit data,
• prefetch, from the memory, second work unit data associated with the second work unit into a standby cache segment of the L1 buffer cache included within the first processing unit, wherein prefetching the second work unit data associated with the second work unit occurs concurrently with at least a portion of the processing of the first work unit by the first processing unit,
• flush, after the processing of the first work unit is complete, the active cache segment of the buffer cache, wherein flushing the active cache segment includes storing, in the memory, the modified first work unit data, and
• generate a message indicating that the modified first work unit data processed by the first work unit can be accessed from the memory.

A non-transitory computer-readable storage medium comprising instructions, that, when executed, cause processing circuitry of a device to:

• identify, prior to completing processing of a first work unit, a second work unit that is expected to be processed by the processing circuitry after the first work unit, each of the first work unit and the second work unit associated with one or more stream fragments, and each of the first work unit and the second work unit specifying a work unit handler for processing the one or more stream fragments;
• process the first work unit, wherein processing the first work unit includes accessing first work unit data associated with the first work unit and stored within a first cache segment of a level one (L1) buffer cache and generating, from the first work unit data, modified first work unit data;
• prefetch, from non-coherent memory, second work unit data associated with the second work unit into a second cache segment of the buffer cache, wherein prefetching the data associated with the second work unit occurs concurrently with at least a portion of the processing of the first work unit;
• flush, after processing the first work unit, the first cache segment of the L1 buffer cache by storing, in the non-coherent memory, the modified first work unit data;
• generate a message indicating that the modified first work unit data associated with the first work unit can be accessed from the non-coherent memory;
• process the second work unit by accessing, from the second cache segment of the L1 buffer cache, the second work unit data associated with the second work unit prefetched into the second cache segment of the L1 buffer cache;
• identify, by the processing unit and prior to completing processing of the second work unit, a third work unit that is expected to be processed by the processing unit after the second work unit;
• and
• prefetch, by the processing unit and from the non-coherent memory, third work unit data associated with the third work unit into the first cache segment of the L1 buffer cache, wherein prefetching the third work unit data associated with the third work unit occurs concurrently with at least a portion of the processing of the second work unit by the processing unit and concurrently with at least a portion of flushing the first cache segment.