System and method for hashed compressed weighting matrix in neural networks

An electronic device, comprising:

• a communication interface;
• a memory storing at least a portion of a neural network;
• and
• at least one processor coupled to the communication interface and the memory, the at least one processor configured to:
• generate a codebook from a weighting matrix of the neural network, wherein, to generate the codebook, the at least one processor is configured to:
• create a plurality of sub-vectors from the weighting matrix;
• generate, by performing a clustering algorithm on the plurality of sub-vectors, a cluster of sub-vectors having similar weight values;
• determine, for the cluster, a centroid for each sub-vector in the cluster;
• and
• store the centroid for each sub-vector in the cluster at a location in the codebook associated with the cluster;
• build an index matrix from the codebook, wherein the index matrix includes one or more index values, and wherein one of the one or more index values represents the location in the codebook associated with the cluster;
• and
• generate a hash table for each input of a plurality of inputs, the hash table being a product of multiplication of the input and the generated codebook.

A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that, when executed by at least one processor of an electronic device, causes the at least one processor to:

• generate a codebook from a weighting matrix of a neural network, wherein the computer readable program code that when executed causes the at least one processor to generate the codebook comprises computer readable program code that when executed causes the at least one processor to:
• create a plurality of sub-vectors from the weighting matrix;
• generate, by performing a clustering algorithm on the plurality of sub-vectors, a cluster of sub-vectors having similar weight values;
• determine, for the cluster, a centroid for each sub-vector in the cluster;
• and
• store the centroid for each sub-vector in the cluster at a location in the codebook associated with the cluster;
• build an index matrix from the codebook, wherein the index matrix includes one or more index values, and wherein one of the one or more index values represents the location in the codebook associated with the cluster;
• and
• generate a hash table for each input of a plurality of inputs, the hash table being a product of multiplication of the input and the generated codebook.

A method for a neural network, wherein at least a portion of the neural network is stored in a memory of an electronic device, the method comprising:

• receiving an input from a vector of inputs;
• determining a table index based on a value of the input;
• and
• retrieving a hash table from a plurality of hash tables, wherein the hash table corresponds to the table index, wherein the hash table is created using a product of multiplication of the value of the input and values of a compressed weighting matrix codebook, and wherein the neural network is trained using an original weighting matrix that is compressed into the compressed weighting matrix codebook.

An electronic device, comprising:

• a communication interface;
• a memory storing at least a portion of a neural network;
• and
• at least one processor coupled to the communication interface and the memory, the at least one processor configured to:
• receive an input from a vector of inputs;
• determine a table index based on a value of the input;
• and
• retrieve a hash table from a plurality of hash tables, wherein the hash table corresponds to the table index, wherein the hash table is created using a product of multiplication of the value of the input and values of a compressed weighting matrix codebook, and wherein the neural network is trained using an original weighting matrix that is compressed into the compressed weighting matrix codebook.

A non-transitory computer readable medium embodying a computer program, the computer program comprising computer readable program code that, when executed by at least one processor of an electronic device having a memory storing at least a portion of a neural network, causes the at least one processor to:

• receive an input from a vector of inputs;
• determine a table index based on a value of the input;
• and
• retrieve a hash table from a plurality of hash tables, wherein the hash table corresponds to the table index, wherein the hash table is created using a product of multiplication of the value of the input and values of a compressed weighting matrix codebook, and wherein the neural network is trained using an original weighting matrix that is compressed into the compressed weighting matrix codebook.