Variable precision and mix type representation of multiple layers in a network

A method comprising:

• exposing, by a processor, embedded cast operations in at least one of a load instruction or a store instruction of a stream of instructions executed by the processor;
• determining, by the processor, for each layer of a multi-layer deep learning neural network (DNN), a target precision level for the cast operations at each layer and data types of a plurality of different data types for the cast operations at each layer, wherein the target precision level for the cast operations at each layer is determined from the plurality of different data types that are used to represent various weights in different layers of the multi-layer DNN, and wherein high precision floating point data is utilized for a first subset of the different layers, low precision floating point data is utilized for a second subset of the different layers, and integer data is utilized for a third subset of the different layers;
• and
• loading, by the processor, the cast operations at the target precision level and the data types determined for the cast operations at each layer of the multi-layer DNN.

A non-transitory machine-readable storage medium having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• exposing, by the one or more processors, embedded cast operations in at least one of a load instruction or a store instruction of a stream of instructions executed by the processor;
• determining, by the one or more processors, for each layer of a multi-layer deep learning neural network (DNN), a target precision level for the cast operations at each layer and data types of a plurality of different data types for the cast operations at each layer, wherein the target precision level for the cast operations at each layer is determined from the plurality of different data types that are used to represent various weights in different layers of the multi-layer DNN, and wherein high precision floating point data is utilized for a first subset of the different layers, low precision floating point data is utilized for a second subset of the different layers, and integer data is utilized for a third subset of the different layers;
• and
• loading, by the one or more processors, the cast operations at the target precision level and the data types determined for the cast operations at each layer of the multi-layer DNN.

A system comprising:

• a memory;
• and
• a processor communicably coupled to the memory, wherein the processor is to:
• expose embedded cast operations in at least one of a load instruction or a store instruction of a stream of instructions;
• determine, for each layer of a multi-layer deep learning neural network (DNN), a target precision level for the cast operations at each layer and data types of a plurality of different data types for the cast operations at each layer, wherein the target precision level for the cast operations at each layer is determined from the plurality of different data types that are used to represent various weights in different layers of the multi-layer DNN, and wherein high precision floating point data is utilized for a first subset of the different layers, low precision floating point data is utilized for a second subset of the different layers, and integer data is utilized for a third subset of the different layers;
• and
• load the cast operations at the target precision level and the data types determined for the cast operations at each layer of the multi-layer DNN.