# US 10,311,863 B2 · Classifying segments of speech based on acoustic features and context > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,863 B2 - Document ID: US10311863B2 - Status: Granted - Kind: B2 - Application number: 15/256,378 - Filed: 2016-09-02 - Published: 2018-03-08 - Granted: 2019-06-04 - Earliest claimed: 2016-09-02 - Assignees: Disney Enterprises, Inc. - Applicants: Disney Enterprises, Inc. - Inventors: Jill Fain Lehman; Nikolas Wolfe; Andre Pereira - Web page: https://patentgrep.com/patent/US10311863B2 ## CPC - G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit - G10L 2015/088: {Word spotting} - G10L 2015/223: {Execution procedure of a spoken command} - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract There is provided a system including a microphone configured to receive an input speech, an analog to digital (A/D) converter configured to convert the input speech to a digital form and generate a digitized speech including a plurality of segments having acoustic features, a memory storing an executable code, and a processor executing the executable code to extract a plurality of acoustic feature vectors from a first segment of the digitized speech, determine, based on the plurality of acoustic feature vectors, a plurality of probability distribution vectors corresponding to the probabilities that the first segment includes each of a first keyword, a second keyword, both the first keyword and the second keyword, a background, and a social speech, and assign a first classification label to the first segment based on an analysis of the plurality of probability distribution vectors of one or more segments preceding the first segment and the probability distribution vectors of the first segment. ## Claims ### Claim 1 A system comprising: a microphone configured to receive an input speech; an analog to digital (A/D) converter configured to convert the input speech from an analog form to a digital form and generate a digitized speech including a plurality of segments having acoustic features; a memory storing an executable code; and a hardware processor executing the executable code to: extract a plurality of acoustic feature vectors from the first segment of the plurality of segments of the digitized speech; determine, based on the plurality of acoustic feature vectors, a plurality of probability distribution vectors corresponding to the probabilities that the first segment of the digitized speech includes each of a first keyword, a second keyword, both the first keyword and the second keyword, a background, and a social speech; assign a first classification label to the first segment based on an analysis of the plurality of probability distribution vectors of one or more segments preceding the first segment and the probability distribution vectors of the first segment, wherein the assigning of the first classification label to the first segment includes distinguishing between the first keyword and an audio event that is one of the social speech and the background; and execute a first action associated with a first command when the first classification label includes the first command. ### Claim 2 (depends on 1) The system of claim 1, wherein the first classification label includes one of the first command, the social speech, and the background. ### Claim 3 (depends on 1) The system of claim 1, wherein executing the first action controls a video game. ### Claim 4 (depends on 1) The system of claim 1, wherein executing the first action controls a robot. ### Claim 5 (depends on 1) The system of claim 1, wherein the hardware processor further executes the executable code to: store the determined plurality of probability distribution vectors of the first segment in the acoustic features database. ### Claim 6 (depends on 1) The system of claim 1, wherein the hardware processor executes the executable code to: determine a second audio event in the first segment of the plurality of segments of the digitized speech is a second keyword based on the acoustic feature vectors extracted from the first segment; analyze a context of the second keyword based on the acoustic features of the one or more segments of the plurality of segments of the digitized speech preceding the first segment; assign a second classification label to the first segment based on the context of the second keyword, the second classification label including one of the first command, a second command, both the first command and the second command, the social speech, and the background. ### Claim 7 (depends on 6) The system of claim 6, wherein, when the second classification label includes one of the second command and both the first command and the second command, the hardware processor further executes the executable code to: execute a second action associated with the second command. ### Claim 8 (depends on 6) The system of claim 6, wherein the first audio event and the second audio event overlap in the first segment of the plurality of segments of the digitized speech. ### Claim 9 A method for use with a system having a hardware processor, and a non-transitory memory, the method comprising: extracting, using the hardware processor, a plurality of acoustic feature vectors from a first segment of a plurality of segments of a digitized speech; determining, using the hardware processor and based on the plurality of acoustic feature vectors, a plurality of probability distribution vectors corresponding to the probabilities that first segment of the digitized speech includes each of a first keyword, a second keyword, both the first keyword and the second keyword, a background, and a social speech; assigning, using the hardware processor, a first classification label to the first segment based on the plurality of probability distribution vectors of one or more segments preceding the first segment and the probability distribution vectors of the first segment, wherein the assigning of the first classification label to the first segment includes distinguishing between the first keyword and an audio event that is one of the social speech and the background; and executing a first action associated with a first command when the first classification label includes the first command. ### Claim 10 (depends on 9) The method of claim 9, wherein the first classification label includes one of the first command, the social speech, and the background. ### Claim 11 (depends on 9) The method of claim 9, wherein executing the first action controls a video game. ### Claim 12 (depends on 9) The method of claim 9, wherein executing the first action controls a robot. ### Claim 13 (depends on 9) The method of claim 9, further comprising: storing the determined plurality of probability distribution vectors of the first segment in the acoustic features database. ### Claim 14 (depends on 9) The method of claim 9, further comprising: determining, using the hardware processor, a second audio event in the first segment of the plurality of segments of the digitized speech is a second keyword based on the acoustic feature vectors extracted from the first segment; analyzing, using the hardware processor, a context of the second keyword based on the acoustic features of the one or more segments of the plurality of segments of the digitized speech preceding the first segment; assigning, using the hardware processor, a second classification label to the first segment based on the context of the second keyword, the second classification label including one of the first command, second a command, both the first command and the second command, the social speech, and the background. ### Claim 15 (depends on 14) The method of claim 14, wherein, when the second classification label includes one of the second command and both the first command and the second command, the method further comprises: executing, using the hardware processor, a second action based on the second command. ### Claim 16 (depends on 14) The method of claim 14, wherein the first audio event and the second audio event overlap in the first segment of the plurality of segments of the digitized speech.