Efficient speech to spikes conversion pipeline for a spiking neural network

A system comprising:

• one or more processors including a neuromorphic processor;
• and
• one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
• obtaining audio data;
• generating frequency domain audio signals that represent the audio data by converting the audio data into a frequency domain;
• mapping the frequency domain audio signals into a set of Mel-frequency bands to obtain Mel-scale frequency audio signals;
• performing a log transformation on the Mel-scale frequency audio signals to obtain log-Mel signals;
• generating spike input comprising a series of spikes for input to a spiking neural network (SNN) model by converting the log-Mel signals to the series of spikes;
• and
• providing the spike input as an input to the SNN model;
• for each hyperparameter configuration of a plurality of sets of hyperparameter configurations:
• applying the sets of hyperparameter configurations to the one or more components of the audio to spikes conversion pipeline;
• training multiple test SNN models using training audio data that is converted to spike inputs using the one or more components of the audio to spikes conversion pipeline;
• and
• testing each test SNN model using testing data, the testing comprising generating result data by determining an accuracy of speech recognition results generated by each test SNN model;
• and
• updating the sets of hyperparameters based on the result data for each set of hyperparameters, thereby reducing amount of processing of the audio data without an adverse impact on accuracy of a conversation corresponding to the audio data.

One or more computer storage media storing instructions that when executed by one or more processors comprising a neuromorphic processor cause the one or more computers to perform the operations comprising:

• obtaining audio data;
• generating frequency domain audio signals that represent the audio data by converting the audio data into a frequency domain;
• mapping the frequency domain audio signals into a set of Mel-frequency bands to obtain Mel-scale frequency audio signals;
• performing a log transformation on the Mel-scale frequency audio signals to obtain log-Mel signals;
• generating spike input comprising a series of spikes for input to a spiking neural network (SNN) model by converting the log-Mel signals to the series of spikes;
• and
• providing the spike input as an input to the SNN model;
• for each hyperparameter configuration of a plurality of sets of hyperparameter configurations:
• applying the sets of hyperparameter configurations to the one or more components of the audio to spikes conversion pipeline;
• training multiple test SNN models using training audio data that is converted to spike inputs using the one or more components of the audio to spikes conversion pipeline;
• and
• testing each test SNN model using testing data, the testing comprising generating result data by determining an accuracy of speech recognition results generated by each test SNN model;
• and
• updating the sets of hyperparameters based on the result data for each set of hyperparameters, thereby reducing amount of processing of the audio data without an adverse impact on accuracy of a conversation corresponding to the audio data.