# US 2019/0318759 A1 · Context-based detection of end-point of utterance > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318759 A1 - Document ID: US20190318759A1 - Status: Published application - Kind: A1 - Application number: 15/951,989 - Filed: 2018-04-12 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Paras Surendra DOSHI; Ayush AGARWAL; Shri PRAKASH - Web page: https://patentgrep.com/patent/US20190318759A1 ## CPC - G10L 15/04: Segmentation; Word boundary detection - G10L 25/87: Detection of discrete points within a voice signal - G10L 15/05: Word boundary detection - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/19: Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 13/00: Speech synthesis; Text to speech systems - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10) - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Detecting an end-point of user's voice command or utterance with high accuracy is critical in automatic speech recognition (ASR)-based human machine interface. If an ASR system incorrectly detects an end-point of utterance and transmits this incomplete sentence to other processing blocks for further processing, it is likely the processed result would lead to incorrect interpretation. A method includes selecting a first semantic network based on context of the audio signal and more accurately detecting the end-point of user's utterance included in the audio signal based on the first semantic network and also based on at least one timeout threshold associated with the first semantic network. ## Claims ### Claim 1 A method for processing an audio signal by an electronic device, the method comprising: receiving, by an automatic speech recognition (ASR) module, an audio signal representing an utterance; selecting a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots; performing, by the ASR module, automatic speech recognition processing on a first portion of the audio signal to generate a first ASR output; determining, by a natural language understanding (NLU) module, the first ASR output corresponds to an incomplete sentence based on the first semantic network; and in response to determination that the first ASR output corresponds to the incomplete sentence, increasing a first timeout threshold associated with the first semantic network. ### Claim 2 (depends on 1) The method of claim 1, further comprising: performing, by the ASR module, automatic speech recognition processing on a second portion of the audio signal to generate a second ASR output; determining, by the NLU module, the second ASR output corresponds to a complete sentence based on the first semantic network; in response to determination that the second ASR output corresponds to the complete sentence, generating a first NLU output; and initiating a first action to be executed on the electronic device, wherein the first action is based on the first NLU output. ### Claim 3 (depends on 2) The method of claim 2, wherein determining the second ASR output corresponds to the complete sentence is based on comparing a subset of the second ASR output with the first semantic network. ### Claim 4 (depends on 3) The method of claim 3, wherein comparing the subset of the second ASR output with the first semantic network comprises identifying a first portion of the subset of the second AR output corresponds to a first slot among the plurality of slots associated with the first semantic network. ### Claim 5 (depends on 1) The method of claim 1, wherein selecting the first semantic network based on the context of the audio signal comprises: performing, by the ASR module, automatic speech recognition processing on a third portion of the audio signal to generate a third ASR output; and determining the context of the audio signal based on the third ASR output. ### Claim 6 (depends on 5) The method of claim 5, wherein selecting the first semantic network based on the context of the audio signal further comprises selecting the first semantic network among a plurality of semantic networks based on the context, wherein at least one timeout threshold is associated with at least one slot of the selected first semantic network. ### Claim 7 (depends on 6) The method of claim 6, wherein at least one semantic network of the plurality of semantic networks corresponds to a recursive transition network (RTN). ### Claim 8 (depends on 1) The method of claim 1, wherein increasing the first timeout threshold is further based on history information related with prior utterances. ### Claim 9 (depends on 1) The method of claim 1, wherein determining the first ASR output corresponds to the incomplete sentence comprises: parsing the first ASR output based on a predetermined grammar; and comparing the parsed first ASR output with the first semantic network. ### Claim 10 (depends on 9) The method of claim 9, wherein comparing the parsed first ASR output with the first semantic network comprises: identifying a first portion of the parsed first AR output corresponds to a first slot among the plurality of slots associated with the first semantic network; and identifying a second portion of the parsed first AR output does not correspond to a second slot among the plurality of slots associated with the first semantic network. ### Claim 11 An apparatus for processing an audio signal, the apparatus comprising: at least one processor; a memory coupled to the at least one processor, the memory including instructions operable to be executed by the at least one processor to perform a set of actions, configuring the at least one processor: to receive an audio signal representing an utterance; to select a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots; to perform automatic speech recognition processing on a first portion of the audio signal to generate a first ASR output; to determine the first ASR output corresponds to an incomplete sentence based on the first semantic network; and in response to determination that the first ASR output corresponds to the incomplete sentence, to increase a first timeout threshold associated with the first semantic network. ### Claim 12 (depends on 11) The apparatus of claim 11, wherein the at least one processor is further configured: to perform automatic speech recognition processing on a second portion of the audio signal to generate a second ASR output; to determine the second ASR output corresponds to a complete sentence based on the first semantic network; and in response to determination that the second ASR output corresponds to the complete sentence, to initiate a first action to be executed, wherein the first action is based on the second ASR output. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the at least one processor is configured to determine the second ASR output corresponds to the complete sentence by comparing a subset of the second ASR output with the first semantic network. ### Claim 14 (depends on 11) The apparatus of claim 11, wherein the at least one processor is configured to select the first semantic network by: performing automatic speech recognition processing on a third portion of the audio signal to generate a third ASR output; and determining the context of the audio signal based on the third ASR output. ### Claim 15 (depends on 11) The apparatus of claim 11, wherein the at least one processor is configured to select the first semantic network by selecting the first semantic network among a plurality of semantic network based on the context, wherein at least one timeout threshold is associated with at least one slot of the selected first semantic network. ### Claim 16 (depends on 11) The apparatus of claim 11, wherein increasing the first timeout threshold is further based on history information related with prior utterances. ### Claim 17 (depends on 11) The apparatus of claim 11, wherein the at least one processor is configured to determine the first ASR output corresponds to the incomplete sentence by: parsing the first ASR output based on a predetermined grammar; and comparing the parsed first ASR output with the first semantic network. ### Claim 18 (depends on 17) The apparatus of claim 17, wherein the at least one processor is configured to compare the parsed first ASR output with the first semantic network by: identifying a first portion of the parsed first AR output corresponds to a first slot among the plurality of slots associated with the first semantic network; and identifying a second portion of the parsed first AR output does not correspond to a second slot among the plurality of slots associated with the first semantic network. ### Claim 19 An apparatus for processing an audio signal, the apparatus comprising: means for receiving an audio signal representing an utterance; means for selecting a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots; means for performing automatic speech recognition processing on a first portion of the audio signal to generate a first ASR output; means for determining the first ASR output corresponds to an incomplete sentence based on the first semantic network; and in response to determination that the first ASR output corresponds to the incomplete sentence, means for increasing a first timeout threshold associated with the first semantic network. ### Claim 20 (depends on 19) The apparatus of claim 19, further comprising: means for performing automatic speech recognition processing on a second portion of the audio signal to generate a second ASR output; means for determining the second ASR output corresponds to a complete sentence based on the first semantic network; and in response to determination that the second ASR output corresponds to the complete sentence, means for initiating a first action to be executed, wherein the first action is based on the second ASR output. ### Claim 21 (depends on 20) The apparatus of claim 20, wherein means for determining the second ASR output corresponds to the complete sentence is based on means for comparing a subset of the second ASR output with the first semantic network. ### Claim 22 (depends on 21) The apparatus of claim 21, wherein means for comparing the subset of the second ASR output with the first semantic network comprises means for identifying a first portion of the subset of the second AR output corresponds to a first slot among the plurality of slots associated with the first semantic network. ### Claim 23 (depends on 19) The apparatus of claim 19, wherein means for selecting the first semantic network based on the context of the audio signal comprises: means for performing automatic speech recognition processing on a third portion of the audio signal to generate a third ASR output; and means for determining the context of the audio signal based on the third ASR output. ### Claim 24 (depends on 19) The apparatus of claim 19, wherein means for determining the first ASR output corresponds to the incomplete sentence comprises: means for parsing the first ASR output based on a predetermined grammar; and means for comparing the parsed first ASR output with the first semantic network. ### Claim 25 (depends on 24) The apparatus of claim 24, wherein means for comparing the parsed first ASR output with the first semantic network comprises: means for identifying a first portion of the parsed first AR output corresponds to a first slot among the plurality of slots associated with the first semantic network; and means for identifying a second portion of the parsed first AR output does not correspond to a second slot among the plurality of slots associated with the first semantic network. ### Claim 26 A non-transitory computer-readable medium comprising instructions that, when executed by a processor, cause the processor to perform operations comprising: receiving an audio signal representing an utterance; selecting a first semantic network based on context of the audio signal, wherein the first semantic network comprises a plurality of slots; performing automatic speech recognition processing on a first portion of the audio signal to generate a first ASR output; determining the first ASR output corresponds to an incomplete sentence based on the first semantic network; and in response to determination that the first ASR output corresponds to the incomplete sentence, increasing a first timeout threshold associated with the first semantic network. ### Claim 27 (depends on 26) The non-transitory computer-readable medium of claim 26, wherein the operations further comprise: performing automatic speech recognition processing on a second portion of the audio signal to generate a second ASR output; determining the second ASR output corresponds to a complete sentence based on the first semantic network; and in response to determination that the second ASR output corresponds to the complete sentence, initiating a first action to be executed, wherein the first action is based on the second ASR output. ### Claim 28 (depends on 27) The non-transitory computer-readable medium of claim 27, wherein determining the second ASR output corresponds to the complete sentence is based on comparing a subset of the second ASR output with the first semantic network. ### Claim 29 (depends on 26) The non-transitory computer-readable medium of claim 26, wherein selecting the first semantic network based on the context of the audio signal comprises: performing automatic speech recognition processing on a third portion of the audio signal to generate a third ASR output; and determining the context of the audio signal based on the third ASR output. ### Claim 30 (depends on 29) The non-transitory computer-readable medium of claim 29, wherein selecting the first semantic network based on the context of the audio signal further comprises selecting the first semantic network among a plurality of semantic network based on the context, wherein at least one timeout threshold is associated with at least one slot of the selected first semantic network.