# US 9,520,142 B2 · Efficient apparatus and method for audio signature generation using recognition history > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,520,142 B2 - Document ID: US9520142B2 - Status: Granted - Kind: B2 - Application number: 14/713,995 - Filed: 2015-05-15 - Published: 2015-11-19 - Granted: 2016-12-13 - Earliest claimed: 2014-05-16 - Assignees: Alphonso Inc. - Applicants: Alphonso Inc. - Inventors: Lampros Kalampoukas; Ashish Chordia; Raghu Srinivas Kodige - Web page: https://patentgrep.com/patent/US9520142B2 ## CPC - H04W 4/023: {using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds} - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - G06F 16/683: using metadata automatically derived from the content - G06F 16/951: Indexing; Web crawling techniques - G06F 16/955: using information identifiers, e.g. uniform resource locators [URL] - G10L 15/06: Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice (G10L15/14 takes precedence) - G10L 19/018: Audio watermarking, i.e. embedding inaudible data in the audio signal - G10L 25/03: characterised by the type of extracted parameters - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/27: characterised by the analysis technique - G10L 25/48: specially adapted for particular use - G10L 25/51: for comparison or discrimination - H04H 60/58: of audio {(determination or detection of speech characteristics in general G10L25/00; speech recognition in general G10L15/00)} - H04R 29/004: {for microphones (H04R29/007 takes precedence)} - H04W 4/021: Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04H: BROADCAST COMMUNICATION (multiplex communication H04J; pictorial communication aspects of broadcast systems H04N) - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract Audio information is monitored by a user device by generating audio signatures of ambient audio content captured by the user device, receiving conditions of the user device that were present when the ambient audio content for each audio signature was captured, performing audio content recognition on the audio signatures by comparing in a query engine the generated audio signatures to audio signatures previously stored in a database associated with the query engine, identifying generated audio signatures that match audio signatures stored in the database within a threshold confidence level, thereby establishing successful audio content recognition, identifying the conditions of the user device that were present for successful audio content recognition, generating scheduling rules regarding the conditions present during successful audio content recognition and controlling when the user device captures audio content and/or generates audio signatures based on the scheduling rules. ## Claims ### Claim 1 A method for monitoring audio information by a user device comprising: (a) generating audio signatures, using an audio signature generator, of ambient audio content captured by the user device; (b) receiving and storing conditions of the user device that were present when the ambient audio content for each audio signature was captured; (c) performing audio content recognition on the audio signatures by comparing in a query engine the generated audio signatures to audio signatures previously stored in a database associated with the query engine; (d) identifying generated audio signatures that match audio signatures stored in the database within a threshold confidence level, thereby establishing successful audio content recognition; (e) identifying the conditions of the user device that were present for successful audio content recognition; (f) generating scheduling rules, in a rule generation unit, regarding the conditions present during successful audio content recognition and providing the scheduling rules to a scheduling logic unit; and (g) controlling when the user device captures audio content and/or generates audio signatures based on the scheduling rules in the scheduling logic unit, wherein the scheduling logic unit allows for capturing of ambient audio content and generation of audio signatures when currently present conditions of the user device match the conditions present during successful audio content recognition, and the scheduling logic unit inhibits capturing of ambient audio content and/or generation of audio signatures when currently present conditions of the user device do not match the conditions present during successful audio content recognition. ### Claim 2 (depends on 1) The method of claim 1 wherein the audio signature generator is in the user device and the query engine and the database are remotely located from the user device in a remote server, the method further comprising: (h) transmitting the generated audio signatures over a communication channel from the user device to the remote server for performing the audio content recognition in the remote server. ### Claim 3 (depends on 2) The method of claim 2 wherein the rule generation unit is located in the remote server and the scheduling logic unit is located in the user device, the method further comprising: (i) transmitting the conditions received in step (b) over the communication channel from the user device to the remote server for use by the rule generation unit in the remote server; and (j) the remote server communicating over the communication channel the scheduling rules to the scheduling logic unit in the user device. ### Claim 4 (depends on 1) The method of claim 1 wherein the conditions include environmental conditions. ### Claim 5 (depends on 1) The method of claim 1 wherein the conditions include time of day. ### Claim 6 (depends on 1) The method of claim 1 wherein the conditions include motion of the user device. ### Claim 7 (depends on 1) The method of claim 1 wherein the user device is a smartphone or tablet. ### Claim 8 (depends on 1) The method of claim 1 wherein the inhibiting interrupts capturing audio content by the user device and/or interrupts generating audio signatures. ### Claim 9 (depends on 1) The method of claim 1 wherein the inhibiting reduces the frequency of capturing audio content by the user device and/or reduces the frequency of generating audio signatures. ### Claim 10 An apparatus for monitoring audio information by a user device, the apparatus comprising: (a) an audio signature generator that generates audio signatures of ambient audio content captured by a user device; (b) a first database that receives and stores conditions of the user device that were present when the ambient audio content for each audio signature was captured; (c) a query engine in communication with a second database associated with the query engine, the query engine configured to: (i) perform audio content recognition on the audio signatures by comparing the generated audio signatures to audio signatures previously stored in the second database, and (ii) identify generated audio signatures that match audio signatures stored in the database within a threshold confidence level, thereby establishing successful audio content recognition; (d) a processor configured to identify the conditions of the user device that were present for successful audio content recognition; (e) a rule generation unit that generates scheduling rules regarding the conditions present during successful audio content recognition; and (g) a scheduling logic unit that receives the generated scheduling rules and controls when the user device captures audio content and/or generates audio signatures based on the scheduling rules, wherein the scheduling logic unit allows for capturing of ambient audio content and generation of audio signatures when currently present conditions of the user device match the conditions present during successful audio content recognition, and the scheduling logic unit inhibits capturing of ambient audio content and/or generation of audio signatures when currently present conditions of the user device do not match the conditions present during successful audio content recognition. ### Claim 11 (depends on 10) The apparatus of claim 10 wherein the audio signature generator is in the user device, the apparatus further comprising: (h) a remote server that includes the query engine and the second database; (i) a transmitter in the user device that transmits the generated audio signatures over a communication channel from the user device to the remote server for performing the audio content recognition in the remote server. ### Claim 12 (depends on 10) The apparatus of claim 10 wherein the rule generation unit is located in the remote server and the scheduling logic unit is located in the user device, and wherein the transmitter in the user device transmits the conditions in the first database over the communication channel from the user device to the remote server for use by the rule generation unit in the remote server, and the remote server communicates over the communication channel the scheduling rules to the scheduling logic unit in the user device. ### Claim 13 (depends on 10) The apparatus of claim 10 wherein the conditions include environmental conditions. ### Claim 14 (depends on 10) The apparatus of claim 10 wherein the conditions include time of day. ### Claim 15 (depends on 10) The apparatus of claim 10 wherein the conditions include motion of the user device. ### Claim 16 (depends on 10) The apparatus of claim 10 wherein the user device is a smartphone or tablet. ### Claim 17 (depends on 10) The apparatus of claim 10 wherein the inhibiting interrupts capturing audio content by the user device and/or interrupts generating audio signatures. ### Claim 18 (depends on 10) The apparatus of claim 10 wherein the inhibiting reduces the frequency of capturing audio content by the user device and/or reduces the frequency of generating audio signatures.