Vocal command recognition
AssigneeSTMicroelectronics S.r.l.
InventorsNunziata Ivana Guarneri, Viviana D'Alto
Life of this patent
Estimating the term…
Abstract
A method to detect a vocal command, the method including: analyzing audio data received from a transducer configured to convert audio into an electric signal and analyzing the data using a first neural network. The method also includes detecting a keyword from the audio data using the first neural network on the edge device, the first neural network being trained to recognize the keyword. The method further includes activating a second neural network after the keyword is identified by the first neural network and analyzing the audio data using the second neural network, the second neural network being trained to recognize a set of vocal commands. The method to detect a vocal command may also include detecting the vocal command word using the second neural network.
Claims20
A method to detect a vocal command word, the method comprising:
- segmenting audio data into a plurality of audio frames, each audio frame having an overlapping portion with an immediate succeeding audio frame;
- analyzing the audio frames using a first neural network on an edge device, the analyzing comprising searching a first part of an audio frame for a first syllable of a keyword in the audio data, searching a second part of the audio frame for a second syllable of the keyword in the audio data, and searching a third part of the audio frame for a filler sound in the audio data;
- detecting the keyword from the audio data using the first neural network, the first neural network having a first processing workload, wherein detecting the keyword comprises identifying the first syllable, the second syllable, and the filler sound based on the analyzing;
- activating a second neural network on the edge device after the keyword is identified by the first neural network, the second neural network having a second processing workload that is greater than the first processing workload, wherein the first neural network and the second neural network form a neural network pipeline as a finite state machine with a first state and a second state, the first neural network activated during the first state, the second neural network deactivated during the first state, the first neural network deactivated during the second state, and the second neural network activated during the second state;
- analyzing the audio data using the second neural network;
- and
- detecting the vocal command word using the second neural network, the second neural network detecting the vocal command word without involvement of a cloud device.
An electronic device comprising:
- a neural network pipeline having the form of a finite state machine, the finite state machine having a first state and a second state,
- wherein a first neural network of the neural network pipeline operates continuously to identify a keyword in an audio data in the first state, a second neural network of the neural network pipeline being inactive during the first state, the first neural network configured to, in the first state:
- analyze the audio data, the first neural network having a first processing workload, and
- detect the keyword from the audio data, and
- wherein the finite state machine is configured to transition from the first state to the second state in response to the first neural network identifying the keyword, wherein the first neural network is deactivated during the second state, the second neural network configured to, in the second state:
- analyze the audio data using the second neural network, the second neural network having a second processing workload that is greater than the first processing workload, and
- detect a vocal command word using the second neural network.
A non-transitory computer-readable storage medium storing a computer program for execution by a processing circuit, the computer program comprising software code portion to cause the processing circuit to implement a method comprising:
- segmenting audio data into a plurality of audio frames, each audio frame having an overlapping portion with an immediate succeeding audio frame;
- analyzing the audio frames using a first neural network on an edge device, the first neural network having a first processing workload, the analyzing comprising searching a first part of an audio frame for a first syllable of a keyword in the audio data, searching a second part of the audio frame for a second syllable of the keyword in the audio data, and searching a third part of the audio frame for a filler sound in the audio data;
- detecting the keyword from the audio data using the first neural network, wherein detecting the keyword comprises identifying the first syllable, the second syllable, and the filler sound based on the analyzing;
- activating a second neural network on the edge device after the keyword is identified by the first neural network, the second neural network having a second processing workload that is greater than the first processing workload, wherein the first neural network and the second neural network form a neural network pipeline as a finite state machine with a first state and a second state, the first neural network activated during the first state, the second neural network deactivated during the first state, the first neural network deactivated during the second state, and the second neural network activated during the second state;
- analyzing the audio data using the second neural network;
- and
- detecting a vocal command word using the second neural network, the second neural network detecting the vocal command word without involvement of a cloud device.
Dependencies are inferred from the claim text.