End-to-end optimization

A system comprising:

• one or more processors;
• and
• a non-transitory computer readable medium storing a plurality of instructions, which when executed, cause the one or more processors to:
• receive a plurality of characteristics of a target artificial intelligence (AI) network;
• apply the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model to;
• and
• receive optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network;
• wherein the plurality of characteristics of a target AI network comprises:
• a plurality of operations that correspond to respective nodes on a computational graph of the target AI network;
• at least one kernel for implementing one or more operations, the kernel comprised of at least one micro-kernel for formatting kernel-operation an input data stream according to a target hardware input data format compatible with underlying target hardware supporting execution of the operation;
• and
• at least one post-processor module embedded into each micro-kernel, wherein each post-processor module converts target hardware output data located in one or more target hardware registers from a first output data format native to the underlying target hardware to a second output data format that matches an input data format of a subsequent micro-kernel.

A non-transitory computer-readable medium comprising instructions for inferring information about a product, the non-transitory computer-readable medium comprising instructions for:

• receiving a plurality of characteristics of a target artificial intelligence (AI) network;
• applying the plurality of characteristics of the target AI network to at least one of a static cost model and a heuristic AI network model to;
• and
• receiving optimized target AI network configuration data from at least one of static cost model and the heuristic AI network model, the optimized target AI network configuration data representative of a subset of the characteristics of the target AI network that minimize a cost function of execution of the target AI network;
• wherein the plurality of characteristics of a target AI network comprises:
• a plurality of operations that correspond to respective nodes on a computational graph of the target AI network;
• at least one kernel for implementing one or more operations, the kernel comprised of at least one micro-kernel for formatting kernel-operation an input data stream according to a target hardware input data format compatible with underlying target hardware supporting execution of the operation;
• and
• at least one post-processor module embedded into each micro-kernel, wherein each post-processor module converts target hardware output data located in one or more target hardware registers from a first output data format native to the underlying target hardware to a second output data format that matches an input data format of a subsequent micro-kernel.