Method and system for code optimization based on statistical data

A system, comprising:

• a hardware processor configured to process data;
• and
• a compiler executing on the processor, wherein the compiler is configured to receive a high-level function in a first high-level code;
• compile the high-level function into a first set of low-level instructions to be executed on a hardware or a simulator;
• generate at least one meta data when the high-level function is compiled into the first set of low-level instructions, wherein the at least one meta data is generated based on a strategy generated by the compiler, wherein the at least one meta data includes information associated with a layer of a machine learning (ML) model being executed on the hardware or the simulator;
• transmit the first set of low-level instructions to the hardware or the simulator;
• receive a plurality of statistical data generated by the hardware or the simulator in response to execution of the first set of low-level instructions;
• determine whether to make changes to the compilation associated with the high-level function in the first high-level code based on the plurality of statistical data, wherein the changes to the compilation includes at least one or more of changing a memory layout, replacing one ML library call with another ML library call, modifying a mapping of data to memory blocks, modifying a precision for an instruction, modifying quantization for the instruction, modifying a processing element to perform a particular operation associated with the instruction, reordering of data dimensions, and a change to methodology to split tensors;
• recompile the high-level function into a second set of low-level instructions to be executed on the hardware or the simulator based on the changes to the compilation;
• and
• transmit the second set of low-level instructions to the hardware or the simulator.

A system, comprising:

• a means for receiving a high-level function in a first high-level code;
• a means for compiling the high-level function into a first set of low-level instructions to be executed on a hardware or a simulator;
• a means for generate generating at least one meta data when the high-level function is compiled into the first set of low-level instructions, wherein the at least one meta data is generated based on a strategy generated by the compiler, wherein the at least one meta data includes information associated with a layer of a machine learning (ML) model being executed on the hardware or the simulator;
• a means for transmitting the first set of low-level instructions to the hardware or the simulator;
• a means for receiving a plurality of statistical data generated by the hardware or the simulator in response to execution of the first set of low-level instructions;
• a means for determining whether to make changes to the compilation associated with the high-level function in the first high-level code based on the plurality of statistical data, wherein the changes to the compilation includes at least one or more of changing a memory layout, replacing one ML library call with another ML library call, modifying a mapping of data to memory blocks, modifying a precision for an instruction, modifying quantization for the instruction, modifying a processing element to perform a particular operation associated with the instruction, reordering of data dimensions, and a change to methodology to split tensors;
• a means for recompiling the high-level function into a second set of low-level instructions to be executed on the hardware or the simulator based on the changes to the compilation;
• and
• a means for transmitting the second set of low-level instructions to the hardware or the simulator.

A compiler implemented method, comprising:

• receiving a high-level function in a first high-level code;
• compiling the high-level function into a first set of low-level instructions to be executed on a hardware or a simulator;
• generating at least one meta data during the compiling, wherein the at least one meta data is generated based on a strategy generated by the compiler, wherein the at least one meta data includes information associated with a first layer of a machine learning (ML) model being executed on the hardware or the simulator, wherein the information includes a mapping of a code section of the first high-level code to the first layer of the ML model;
• transmitting the first set of low-level instructions to the hardware or the simulator;
• receiving a plurality of statistical data generated by the hardware or the simulator in response to execution of the first set of low-level instructions, and wherein the plurality of statistical data includes cycle counts associated with a second layer of the ML model;
• determining whether to make changes to the compilation associated with the high-level function in the first high-level code based on the plurality of statistical data;
• recompiling the high-level function into a second set of low-level instructions to be executed on the hardware or the simulator based on the changes to the compilation;
• and
• transmitting the second set of low-level instructions to the hardware or the simulator.

A system, comprising:

• a hardware processor configured to process data;
• and
• a compiler executing on the processor, wherein the compiler is configured to receive a high-level function in a first high-level code;
• compile the high-level function into a first set of low-level instructions to be executed on a hardware or a simulator;
• generate at least one meta data when the high-level function is compiled into the first set of low-level instructions, wherein the at least one meta data is generated based on a strategy generated by the compiler, wherein the at least one meta data includes information associated with a first layer of a machine learning (ML) model being executed on the hardware or the simulator, wherein the information includes a mapping of a code section of the first high-level code to the first layer of the ML model;
• transmit the first set of low-level instructions to the hardware or the simulator;
• receive a plurality of statistical data generated by the hardware or the simulator in response to execution of the first set of low-level instructions, and wherein the plurality of statistical data includes cycle counts associated with a second layer of the ML model;
• determine whether to make changes to the compilation associated with the high-level function in the first high-level code based on the plurality of statistical data;
• recompile the high-level function into a second set of low-level instructions to be executed on the hardware or the simulator based on the changes to the compilation;
• and
• transmit the second set of low-level instructions to the hardware or the simulator.

A system, comprising:

• a means for receiving a high-level function in a first high-level code;
• a means for compiling the high-level function into a first set of low-level instructions to be executed on a hardware or a simulator;
• a means for generating at least one meta data when the high-level function is compiled into the first set of low-level instructions, wherein the at least one meta data is generated based on a strategy generated by the compiler, wherein the at least one meta data includes information associated with a first layer of a machine learning (ML) model being executed on the hardware or the simulator, wherein the information includes a mapping of a code section of the first high-level code to the first layer of the ML model;
• a means for transmitting the first set of low-level instructions to the hardware or the simulator;
• a means for receiving a plurality of statistical data generated by the hardware or the simulator in response to execution of the first set of low-level instructions, and wherein the plurality of statistical data includes cycle counts associated with a second layer of the ML model;
• a means for determining whether to make changes to the compilation associated with the high-level function in the first high-level code based on the plurality of statistical data;
• a means for recompiling the high-level function into a second set of low-level instructions to be executed on the hardware or the simulator based on the changes to the compilation;
• and
• a means for transmitting the second set of low-level instructions to the hardware or the simulator.