# US 10,121,472 B2 · Audio buffer catch-up apparatus and method with two microphones > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,472 B2 - Document ID: US10121472B2 - Status: Granted - Kind: B2 - Application number: 15/016,748 - Filed: 2016-02-05 - Published: 2016-08-18 - Granted: 2018-11-06 - Earliest claimed: 2015-02-13 - Assignees: Knowles Electronics, LLC - Applicants: Knowles Electronics, LLC - Inventors: Ramanujapuram Raghuvir - Web page: https://patentgrep.com/patent/US10121472B2 ## CPC - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/32: Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems - G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) - H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)} - G10L 2015/088: {Word spotting} - G10L 2015/223: {Execution procedure of a spoken command} - H04R 2201/003: Mems transducers or their use - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A first microphone is operated in a low power sensing mode, and a buffer at the first microphone is used to temporarily store at least some of the phrase. Subsequently the first microphone is deactivated, then the first microphone is re-activated to operate in normal operating mode where the buffer is no longer used to store the phrase. The first microphone forms first data that does not include the entire phrase. A second microphone is maintained in a deactivated mode until the trigger portion is detected in the first data, and when the trigger portion is detected, the second microphone is caused to operate in normal operating mode where no buffer is used. The second microphone forms second data that does not include the entire phrase. A first electronic representation of the phrase as received at the first microphone and a second electronic representation of the phrase as received at the second microphone are formed from selected portions of the first data and the second data. ## Claims ### Claim 1 An audio processor configured to form an electronic representation of an uttered phrase, the processor comprising: a microphone interface; the processor configured to: receive, via the microphone interface, buffered data derived from a first segment of the uttered phrase by a first microphone; receive, via the microphone interface, first unbuffered data derived from a second segment of the uttered phrase by the first microphone, the first segment different than the second segment; receive, via the microphone interface, second unbuffered data derived from less than the entire uttered phrase by a second microphone; and form the electronic representation of the uttered phrase based on the buffered data, the first unbuffered data, and the second unbuffered data. ### Claim 2 (depends on 1) The processor of claim 1, further configured to: provide a second microphone enablement signal at the microphone interface after receiving at least some of the buffered data at the microphone interface; and receive the second unbuffered data at the microphone interface after providing the second microphone enablement signal. ### Claim 3 (depends on 2) The processor of claim 2, further configured to: detect speech in the buffered data; and provide the second microphone enablement signal after detecting speech in the buffered data. ### Claim 4 (depends on 1) The processor of claim 1, further configured to: receive the second unbuffered data at the microphone interface after receiving the buffered data; provide a first microphone clock signal to the microphone interface after receiving the buffered data; and receive the first unbuffered data at the microphone interface after providing the first microphone clock signal. ### Claim 5 (depends on 4) The processor of claim 4, further configured to: provide a second microphone enablement signal at the microphone interface after receiving the buffered data at the microphone interface; and receive the second unbuffered data at the microphone interface after providing the second microphone enablement signal. ### Claim 6 (depends on 5) The processor of claim 5, further configured to: detect speech in the buffered data received at the microphone interface; provide the second microphone enablement signal after detecting speech. ### Claim 7 (depends on 4) The processor of claim 4, further configured to: calibrate the first microphone and second microphone based on common information received from the first microphone and the second microphone at the microphone interface. ### Claim 8 (depends on 1) The processor of claim 1, further configured to: provide a first microphone deactivation signal and subsequently provide a first microphone reactivation signal at the microphone interface while receiving the second unbuffered data at the microphone interface; and receive the first unbuffered data at the microphone interface after providing the first microphone reactivation signal. ### Claim 9 (depends on 8) The processor of claim 8, further configured to: provide the first microphone deactivation signal after a delay period occurring when the second unbuffered data is received at the microphone interface. ### Claim 10 (depends on 1) The processor of claim 1, further configured to: form a second electronic representation of the entire uttered phrase based on the buffered data and based on the second unbuffered data. ### Claim 11 A processor configured to form multiple electronic representations of a phrase that is uttered by a user and received at different microphones, the processor comprising: a microphone interface; the processor configured to: receive buffered first data from a first microphone that is being operated in a low power sensing mode, the buffered first data having been buffered at the first microphone; determine whether speech is present in the buffered first data received from the first microphone; activate a second microphone when speech is determined to be present in the buffered first data; subsequently receive unbuffered second data from the second microphone in real time, the unbuffered second data not having been buffered at the second microphone; deactivate the first microphone after a delay period while continuing to receive the unbuffered second data in real time from the second microphone, the deactivation causing the first microphone to fail to sense a part of the phrase, the unbuffered second data including the part of the phrase not sensed by the first microphone; re-activate the first microphone, the re-activation causing the first microphone to operate in a normal operating mode, wherein the first microphone produces unbuffered first data when in normal operating mode; form a first electronic representation of the entire phrase, the first electronic representation being assembled from the buffered first data, the unbuffered second data, and the unbuffered first data; and form a second electronic representation of the entire phrase, the second electronic representation being assembled from the buffered first data and the unbuffered second data. ### Claim 12 (depends on 11) The processor of claim 11, wherein the phrase includes a trigger and a command, and wherein the first data is different than the second data, the processor further configured to: determine whether speech is present by determining whether a trigger is present in the buffered first data received from the first microphone, activate the second microphone when the trigger is determined to be present in the buffered first data. ### Claim 13 (depends on 11) The processor of claim 11, wherein the first microphone and the second microphone are micro electro mechanical system (MEMS) microphones. ### Claim 14 (depends on 11) The processor of claim 11, wherein the first microphone includes a voice activity detector. ### Claim 15 (depends on 11) The processor of claim 11, wherein the second microphone is a pulse density modulation (PDM) microphone. ### Claim 16 A system of forming an electronic representation of an uttered phrase received at different microphones, the system comprising: a first microphone that includes a buffer, wherein the buffer at the first microphone temporarily stores data corresponding to at least some of the phrase; a second microphone; a control apparatus coupled to the first microphone and the second microphone, the control apparatus having an output, the first microphone forming first data including buffered data corresponding to a first segment of the phrase and unbuffered data corresponding to a second segment of the phrase, the control apparatus further configured to cause the second microphone to provide unbuffered second data, the second data corresponding to less than the entire phrase, the control apparatus configured to form a first electronic representation of the entire phrase from selected portions of the first data and the second data, the first electronic representation transmitted at the output. ### Claim 17 (depends on 16) The system of claim 16, wherein: the first data is different than the second data, the first microphone includes a voice activity detector and is operated in a low power sensing mode when the buffer at the first microphone temporarily stores data, the control apparatus configured to maintain the second microphone in a deactivated mode until the control apparatus detects speech in the buffered data, and when speech is detected, cause the second microphone to provide unbuffered second data to the control apparatus. ### Claim 18 (depends on 17) The system of claim 17, wherein: after activating the second microphone, the control apparatus configured to deactivate the first microphone and then re-activate the first microphone to operate in a mode where data is not buffered, the first data including buffered data and unbuffered data from the first microphone, the first data corresponding to less than the entire phrase, the control apparatus configured to form the first electronic representation of the entire phrase from the first data and at least a portion of the second data, and the control apparatus configured to form a second electronic representation of the entire phrase from the buffered data and the second data, the control apparatus configured to transmit the second electronic representation at the output. ### Claim 19 (depends on 17) The system of claim 17, wherein the first microphone and the second microphone are micro electro mechanical system (MEMS) microphones. ### Claim 20 (depends on 17) The system of claim 17, wherein the second microphone is a pulse density modulation (PDM) microphone. ### Claim 21 (depends on 16) The system of claim 16, wherein the control apparatus is configured to maintain the second microphone in a deactivated mode until speech is detected in the first data. ### Claim 22 (depends on 16) The system of claim 16, wherein the control apparatus is configured to form a second electronic representation of the entire phrase from selected portions of the buffered data and the second data.