Statically Generated Compiled Representations for Processing Data in Neural Networks

A method for generating a compiled representation in an electronic device that includes a local cache memory;

• a memory, the memory storing an input matrix A, input matrix A having values to be used when processing instances of input data through the neural network;
• and a processor, the method comprising:
• generating, by the processor, a compiled representation that includes values for acquiring data from input matrix A when processing instances of input data through the neural network, the values including a base address in input matrix A for each thread from among a number of threads and relative offsets, the relative offsets being distances between elements of input matrix A to be processed by the threads;
• and
• storing, by the processor, in the local cache memory, the compiled representation including the base address for each thread and the relative offsets.

An electronic device, comprising:

• a memory storing input matrix A and input matrix B, input matrix A and input matrix B having respective values to be used when processing instances of input data through a neural network;
• a local cache memory that stores a compiled representation, the compiled representation including values for acquiring data from input matrix A when processing instances of input data through the neural network, the values including a base address in input matrix A for each thread from among a number of threads and relative offsets, the relative offsets being distances between elements of input matrix A to be processed by the threads;
• and
• a processor configured to:
• acquire, from the local cache memory, the compiled representation;
• and
• process input matrix A by, for each thread from among the number of threads, using the compiled representation to identify locations in memory from which values in elements of input matrix A are to be acquired to be used, along with values from elements at corresponding locations from input matrix B, as inputs for one or more general matrix multiplication (GEMM) operations by that thread.

A method for processing instances of input data through a neural network in an electronic device that includes a local cache memory that stores a compiled representation;

• a memory storing input matrix A and input matrix B, input matrix A and input matrix B having respective values to be used when processing instances of input data through a neural network;
• and a processor, the method comprising:
• acquiring, by the processor, from the local cache memory, the compiled representation, the compiled representation including values for acquiring data from input matrix A when processing instances of input data through the neural network, the values including a base address in input matrix A for each thread from among a number of threads and relative offsets, the relative offsets being distances between elements of input matrix A to be processed by the threads;
• and
• processing, by the processor, input matrix A by, for each thread from among the number of threads, using the compiled representation to identify locations in memory from which values in elements of input matrix A are to be acquired to be used, along with values from elements at corresponding locations from input matrix B, as inputs for one or more general matrix multiplication (GEMM) operations by that thread.