Voice recognition using accelerometers for sensing bone conduction

A method of word recognition spoken by a user, comprising:

• sensing, using an accelerometer in a housing, a plurality of reference audio vibrations that are conducted from a plurality of different bones of the user's body, the plurality of reference audio vibrations being sensed in response to the user speaking a plurality of different selected words, respectively, the plurality of different selected words being spoken while the housing is positioned at a plurality of different locations, respectively, on the user's body;
• outputting, from the accelerometer and to a processor, reference data based on the sensed plurality of reference audio vibrations and location data indicating the plurality of different locations;
• storing, by the processor, the reference data in correspondence with the plurality of different selected words and the location data;
• training, by the processor, a classification model using the reference data, the plurality of different selected words, and the location data, the training including:
• performing signal convolution on the reference audio vibrations conducted from the bone of the user;
• modifying the output of the signal convolution with a non-linearity function;
• and
• performing a scale bias and normalization on the modified signal from the non-linearity function;
• sensing, using the accelerometer, sample audio vibrations that are conducted from a bone of the user while the user wears the housing after training the classification model;
• outputting, from the accelerometer to the processor, a sample signal corresponding to the sensed sample audio vibrations;
• pooling the sample signal on which scale bias and normalization were performed;
• performing a second signal convolution on the vibrations conducted from the bone of the user after the pooling step;
• modifying the output of the second signal convolution with a second non-linearity function;
• performing a second scale bias and normalization on the modified signal from the non-linearity function
• pooling the output of the second scale bias and normalization step;
• applying a dense function to the twice pooled signal;
• and
• performing a third non-linearity function to the bone conduction data after the dense function;
• calculating a label value using the sample signal as input against the classification model;
• determining the sample signal is a match to a word of the plurality of different selected stored words based on the location data and if the label value is within a threshold value;
• and
• audibly outputting, from the speaker included in the housing, the word of the plurality of different selected words if the sample signal is within the threshold value.

A method of performing word recognition spoken by a user, comprising:

• sensing a reference set of vibrations conducted from a bone of the user using an accelerometer in a housing, the reference set of vibrations being sensed in response to the user speaking a selected word, the selected word being spoken while the housing is positioned at a location on the user's body, wherein a speaker is positioned in the housing;
• outputting, from the accelerometer to a processor, a reference signal based on the sensed reference set of vibrations and a location signal indicating the location on the user's body;
• storing the reference signal and the location signal in a memory;
• receiving an input providing an identity of a word that corresponds to the reference signal and the location signal;
• storing the word as a match to the reference signal and the location signal in the memory;
• repeating the prior steps of sensing, outputting, storing, receiving and storing for a plurality of reference sets to store a plurality of different words, the plurality of reference sets being sensed in response to the user speaking a plurality of different selected words, respectively, the plurality of different selected words being spoken while the housing is positioned at a plurality of different locations, respectively, on the user's body;
• training a classification model using the stored plurality of reference sets, including:
• performing signal convolution on the vibrations conducted from the bone of the user;
• modifying the output of the signal convolution with a non-linearity function;
• and
• performing a scale bias and normalization on the modified signal from the non-linearity function;
• sensing a sample set of vibrations conducted from a bone of the user while the user wears the housing after the classification model has been trained;
• outputting, from the accelerometer to the processor, a sample signal based on the sensed sample set of vibrations;
• pooling the sample signal on which scale bias and normalization were performed;
• performing a second signal convolution on the vibrations conducted from the bone of the user after the pooling step;
• modifying the output of the second signal convolution with a second non-linearity function;
• performing a second scale bias and normalization on the modified signal from the non-linearity function pooling the output of the second scale bias and normalization step;
• applying a dense function to the twice pooled signal;
• and
• performing a third non-linearity function to the bone conduction data after the dense function;
• calculating a label value using the sample signal as input against the classification model;
• determining the sample signal is a match to a stored word if the label value is within a threshold value;
• and
• audibly outputting the stored word from the speaker if the sample signal is a match.

A device, comprising:

• a housing;
• an accelerometer in the housing;
• a speaker in the housing;
• a power supply in the housing coupled to the accelerometer;
• and
• a processor in the housing coupled to the accelerometer, the processor configured to:
• store a plurality of sets of collected accelerometer signals of the accelerometer and a plurality of locations of the accelerometer on a user's body, the plurality of sets of collected accelerometer signals being sensed in response to the user speaking a plurality of different selected words, respectively, the plurality of different selected words being spoken while the accelerometer is positioned at the plurality of different locations, respectively, on the user's body;
• train a classification model using the reference data, the plurality of different selected words, and the location data, including:
• performing signal convolution on vibrations conducted from a bone of the user;
• modifying the output of the signal convolution with a non-linearity function;
• and
• performing a scale bias and normalization on the modified signal from the non-linearity function;
• receive sample data generated by the accelerometer while the user wears the housing after the classification model has been trained;
• pooling the sample signal on which scale bias and normalization were performed;
• performing a second signal convolution on the vibrations conducted from the bone of the user after the pooling step;
• modifying the output of the second signal convolution with a second non-linearity function;
• performing a second scale bias and normalization on the modified signal from the non-linearity function
• pooling the output of the second scale bias and normalization step;
• applying a dense function to the twice pooled signal;
• and
• performing a third non-linearity function to the bone conduction data after the dense function;
• determine a spoken word with the function and the plurality of locations of the accelerometer on the user's body in response to a second collected accelerometer signal;
• and
• cause the speaker to audibly output the word in response to the sample data being determined to match the word of the plurality of different selected words.