Method and system for multiple time resolution audio processing

A computer program product configured to enable voice control, the computer program product comprising one or more non-transitory computer-readable media, having instructions stored thereon that when executed by one or more processors cause the one or more processors, individually or in combination, to perform a method comprising:

• transforming, using a short-time Fourier transform (STFT) applied to data in each of a plurality of windows aligned across each input channel of a multichannel audio stream, the multichannel audio stream into a complex valued frequency-domain representation,
• wherein for a current one of the plurality of windows, the method comprises:
• updating a first complex-valued covariance matrix corresponding to a slowly-adapting beamformer and forming a single-channel denoised estimate for each frequency band in the STFT;
• calculating a voice activity detection (VAD) estimate for each frequency band in the STFT by comparing a magnitude of the single-channel denoised estimate to a magnitude of each input channel of the multichannel audio stream;
• and
• selectively updating or refraining from updating, responsive to the VAD estimate respectively indicating a presence or an absence of speech, a second complex-valued covariance matrix corresponding to a quickly-adapting beamformer;
• and
• controlling, by a hardware processor, a voice user interface based device to perform a user perceptible action, responsive to an output of at least the quickly-adapting beamformer.

A voice control system, comprising:

• one or more memories, individually or in combination, having instructions;
• one or more processors each coupled to at least one of the one or more memories and configurable to execute the instructions to
• transform, using a short-time Fourier transform (STFT) applied to data in each of a plurality of windows aligned across each input channel of a multichannel audio stream, the multichannel audio stream into a complex valued frequency-domain representation,
• wherein for a current one of the plurality of windows, the one or more processors are configurable to execute the instructions to:
• update a first complex-valued covariance matrix corresponding to a slowly-adapting beamformer and forming a single-channel denoised estimate for each frequency band in the STFT;
• calculate a voice activity detection (VAD) estimate for each frequency band in the STFT by comparing a magnitude of the single-channel denoised estimate to a magnitude of each input channel of the multichannel audio stream;
• and
• selectively update or refraining from updating, responsive to the VAD estimate respectively indicating a presence or an absence of speech, a second complex-valued covariance matrix corresponding to a quickly-adapting beamformer;
• and
• control a voice user interface based device to perform a user perceptible action, responsive to an output of at least the quickly-adapting beamformer.