Acoustic system and method based gesture detection using spiking neural networks

A system, comprising:

• a memory storing instructions;
• one or more communication interfaces;
• and
• one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:
• transmit, via a plurality of speakers, a plurality of modulated signals to a user;
• receive, via a plurality of microphones, a plurality of reflected signals from the user, in response to the plurality of transmitted modulated signals;
• process, via a Channel Impulse Response (CIR) estimator, the plurality of transmitted modulated signals and the plurality of reflected signals using a sparsity prior serving as a constraint to obtain a plurality of CIR images;
• and
• recognize, via a Spiking Neural Network (SNN), a gesture performed by the user based on the plurality of CIR images,
• wherein the Spiking Neural Network is obtained by:
• training a Convolutional Neural Network (CNN) using training data comprising a plurality of CIR images corresponding to one or more users to obtain a trained CNN;
• quantizing the trained CNN to obtain a quantized CNN;
• and
• converting the quantized CNN to the SNN, wherein the quantized CNN is converted to the SNN by performing an approximate matching of a corresponding output of an CNN neuron comprised in the CNN to a firing rate of a spiking neuron comprised in the SNN.

One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:

• transmitting, via a plurality of speakers, a plurality of modulated signals to a user;
• receiving, via a plurality of microphones, a plurality of reflected signals from the user, in response to the plurality of transmitted modulated signals;
• processing, via a Channel Impulse Response (CIR) estimator, the plurality of transmitted modulated signals and the plurality of reflected signals using a sparsity prior serving as a constraint to obtain a plurality of CIR images;
• and
• recognizing, via a Spiking Neural Network (SNN), a gesture performed by the user based on the plurality of CIR images,
• wherein the Spiking Neural Network is obtained by:
• training a Convolutional Neural Network (CNN) using training data comprising a plurality of CIR images corresponding to one or more users to obtain a trained CNN;
• quantizing the trained CNN to obtain a quantized CNN;
• and
• converting the quantized CNN to the SNN, wherein the quantized CNN is converted to the SNN by performing an approximate matching of a corresponding output of an CNN neuron comprised in the CNN to a firing rate of a spiking neuron comprised in the SNN.