Apparatus and method for scheduling inference tasks

An apparatus comprising:

• a plurality of compute units configurable to operate in a single instruction multiple data (SIMD) mode or a single instruction multiple thread (SIMT) mode;
• and
• control circuitry coupled to the plurality of compute units and configured to:
• determine, during execution of machine learning operations comprising a plurality of phases, whether a current phase is to be executed in the SIMD mode or the SIMT mode,
• reconfigure at least one compute unit of the plurality of compute units between the SIMD mode and the SIMT mode responsive to the determination;
• and
• dispatch instructions of the current phase to the at least one compute unit for execution in accordance with the determined mode.

A method comprising:

• determining, during execution of machine learning operations on a plurality of compute units of an apparatus, whether a current phase of the machine learning operations is to be executed in a single instruction multiple data (SIMD) mode or a single instruction multiple thread (SIMT) mode, the plurality of compute units configurable to operate in the SIMD mode or the SIMT mode, and the machine learning operations comprising a plurality of phases including the current phase;
• reconfigure at least one compute unit of the plurality of compute units between the SIMD mode and the SIMT mode responsive to the determination;
• and
• dispatch instructions of the current phase to the at least one compute unit for execution in accordance with the determined mode.

A non-transitory machine-readable medium having program code stored thereon which, when executed by a machine, causes the machine to perform:

• determining, during execution of machine learning operations on a plurality of compute units of an apparatus, whether a current phase of the machine learning operations is to be executed in a single instruction multiple data (SIMD) mode or a single instruction multiple thread (SIMT) mode, the plurality of compute units configurable to operate in the SIMD mode or the SIMT mode, and the machine learning operations comprising a plurality of phases including the current phase;
• reconfigure at least one compute unit of the plurality of compute units between the SIMD mode and the SIMT mode responsive to the determination;
• and
• dispatch instructions of the current phase to the at least one compute unit for execution in accordance with the determined mode.