# US 10,373,617 B2 · Reducing the need for manual start/end-pointing and trigger phrases

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,373,617 B2
- Document ID: US10373617B2
- Status: Granted
- Kind: B2
- Application number: 15/656,793
- Filed: 2017-07-21
- Published: 2018-01-11
- Granted: 2019-08-06
- Earliest claimed: 2014-05-30
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Philippe P. Piernot; Justin Binder
- Web page: https://patentgrep.com/patent/US10373617B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04W 4/025: {using location based information parameters}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 17/00: Speaker identification or verification techniques
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Systems and processes for selectively processing and responding to a spoken user input are provided. In one example, audio input containing a spoken user input can be received at a user device. The spoken user input can be identified from the audio input by identifying start and end-points of the spoken user input. It can be determined whether or not the spoken user input was intended for a virtual assistant based on contextual information. The determination can be made using a rule-based system or a probabilistic system. If it is determined that the spoken user input was intended for the virtual assistant, the spoken user input can be processed and an appropriate response can be generated. If it is instead determined that the spoken user input was not intended for the virtual assistant, the spoken user input can be ignored and/or no response can be generated.

## Claims

### Claim 1

A method for operating a virtual assistant on an electronic device, the method comprising:
receiving, at the electronic device, an audio input;
monitoring the audio input to identify a first spoken user input, wherein the first spoken user input comprises a user request;
identifying the first spoken user input in the audio input;
determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a direction of the user's gaze when the first spoken user input was received, wherein the determining comprises:
calculating a likelihood score that the virtual assistant should provide an audible response to the first spoken user input based on the contextual information associated with the first spoken user input, wherein the audible response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the first spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the first spoken user input was received;
in response to a determination to respond to the first spoken user input:
generating the audible response to the first spoken user input; and
monitoring the audio input to identify a second spoken user input; and
in response to a determination not to respond to the first spoken user input, monitoring the audio input to identify the second spoken user input without generating the audible response to the first spoken user input.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input excludes identifying one or more predetermined words at a start of the first spoken user input.

### Claim 3 (depends on 1)

The method of claim 1, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input excludes identifying a physical or virtual button input received prior to receiving the first spoken user input.

### Claim 4 (depends on 1)

The method of claim 1, wherein generating the audible response to the first spoken user input comprises one or more of:
performing speech-to-text conversion on the first spoken user input;
determining a user intent based on the first spoken user input;
determining a task to be performed based on the first spoken user input;
determining a parameter for the task to be performed based on the first spoken user input;
performing the task to be performed;
displaying a text response to the first spoken user input; and
outputting an audio response to the first spoken user input.

### Claim 5 (depends on 1)

The method of claim 1, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the first spoken user input.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input comprises:
comparing the likelihood score to a threshold value.

### Claim 7 (depends on 6)

The method of claim 6, wherein the contextual information comprises an elapsed time between receiving the first spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 8 (depends on 6)

The method of claim 6, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the first spoken user input.

### Claim 9 (depends on 6)

The method of claim 6, wherein the contextual information comprises a distance between a user and the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to the distance being greater than a threshold distance; and
increasing the likelihood score in response to the distance being less than the threshold distance.

### Claim 10 (depends on 6)

The method of claim 6, wherein the contextual information comprises an orientation of the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 11 (depends on 6)

The method of claim 6, wherein the contextual information comprises an orientation between the user and the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and
decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 12 (depends on 6)

The method of claim 6, wherein the contextual information comprises an indication of whether the first spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the first spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the first spoken user input was not recognized by the automatic speech recognizer.

### Claim 13 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 14 (depends on 6)

The method of claim 6, wherein the contextual information comprises a length of the first spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the length of the first spoken user input less than a threshold length; and
decreasing the likelihood score in response to the length of the first spoken user input being greater than the threshold length.

### Claim 15 (depends on 6)

The method of claim 6, wherein the contextual information comprises an identification of a speaker of the first spoken user input.

### Claim 16 (depends on 6)

The method of claim 6, wherein the contextual information comprises a time the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the time being within a predetermined set of times; and
decreasing the likelihood score in response to the time not being within the predetermined set of times.

### Claim 17 (depends on 6)

The method of claim 6, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the first spoken user input was received.

### Claim 18 (depends on 6)

The method of claim 6, wherein the contextual information comprises an expectation of receiving input from the user, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the expectation of receiving input from the user indicating that input was expected to be received from the user; and
decreasing the likelihood score in response to the expectation of receiving input from the user indicating that input was not expected to be received from the user.

### Claim 19 (depends on 6)

The method of claim 6, wherein the contextual information comprises an indication of whether the electronic device is being held when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the first spoken user input was received.

### Claim 20 (depends on 6)

The method of claim 6, wherein the contextual information comprises an operating state of the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the operating state of the electronic device being one of a set of predetermined operating states; and
decreasing the likelihood score in response to the operating state of the electronic device not being one of the set of predetermined operating states.

### Claim 21 (depends on 6)

The method of claim 6, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 22 (depends on 6)

The method of claim 6, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the first spoken user input was received.

### Claim 23 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and content being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 24 (depends on 6)

The method of claim 6, wherein the contextual information comprises a position of the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 25 (depends on 6)

The method of claim 6, wherein the contextual information comprises a gesture being performed by the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the gesture being one of a predetermined set of gestures; and
decreasing the likelihood score in response to the gesture not being one of the predetermined set of gestures.

### Claim 26 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 27 (depends on 6)

The method of claim 6, wherein the contextual information comprises a location of the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to the location being one of a predetermined set of locations; and
increasing the likelihood score in response to the location not being one of the predetermined set of locations.

### Claim 28 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input an application being run by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input device based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 29 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and a previous contact, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous contact semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous contact semantic threshold value.

### Claim 30 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and a previous email, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous email semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous email semantic threshold value.

### Claim 31 (depends on 6)

The method of claim 6, wherein the contextual information comprises a semantic relationship between the first spoken user input and a previous SMS message, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous SMS message semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous SMS semantic threshold value.

### Claim 32 (depends on 6)

The method of claim 6, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.

### Claim 33 (depends on 6)

The method of claim 6, wherein the contextual information comprises a user setting, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the user setting being one of a predetermined set of user settings; and
decreasing the likelihood score in response to the user setting not being one of the predetermined set of user settings.

### Claim 34 (depends on 6)

The method of claim 6, wherein the contextual information comprises an amount of light sensed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the amount of light being greater than a threshold amount of light; and
decreasing the likelihood score in response to the amount of light being less than the threshold amount of light.

### Claim 35 (depends on 6)

The method of claim 6, wherein the contextual information comprises calendar data, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises decreasing the likelihood score in response to the calendar data indicating that the user is occupied at the time that the first spoken user input was received.

### Claim 36

A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the device to:
receive an audio input;
monitor the audio input to identify a first spoken user input, wherein the first spoken user input comprises a user request;
identify the first spoken user input in the audio input;
determine whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a direction of the user's gaze when the first spoken user input was received, wherein the determining comprises:
calculating a likelihood score that the virtual assistant should provide an audible response to the first spoken user input based on the contextual information associated with the first spoken user input, wherein the audible response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the first spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the first spoken user input was received;
responsive to a determination to respond to the first spoken user input:
generate the audible response to the first spoken user input; and
monitor the audio input to identify a second spoken user input; and
responsive to a determination not to respond to the first spoken user input, monitor the audio input to identify the second spoken user input without generating the audible response to the first spoken user input.

### Claim 37

A system comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving an audio input;
monitoring the audio input to identify a first spoken user input, wherein the first spoken user input comprises a user request;
identifying the first spoken user input in the audio input;
determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input, wherein the contextual information comprises a direction of the user's gaze when the first spoken user input was received, wherein the determining comprises:
calculating a likelihood score that the virtual assistant should provide an audible response to the first spoken user input based on the contextual information associated with the first spoken user input, wherein the audible response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the first spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the first spoken user input was received;
responsive to a determination to respond to the first spoken user input:
generating the audible response to the first spoken user input; and
monitoring the audio input to identify a second spoken user input; and
responsive to a determination not to respond to the first spoken user input, monitoring the audio input to identify the second spoken user input without generating the audible response to the first spoken user input.

### Claim 38 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input excludes identifying a physical or virtual button input received prior to receiving the first spoken user input.

### Claim 39 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the first spoken user input.

### Claim 40 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an elapsed time between receiving the first spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 41 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the first spoken user input.

### Claim 42 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an orientation of the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 43 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an orientation between the user and the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and
decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 44 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an indication of whether the first spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the first spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the first spoken user input was not recognized by the automatic speech recognizer.

### Claim 45 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 46 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an identification of a speaker of the first spoken user input.

### Claim 47 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the first spoken user input was received.

### Claim 48 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an indication of whether the electronic device is being held when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the first spoken user input was received.

### Claim 49 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 50 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the first spoken user input was received.

### Claim 51 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a semantic relationship between the first spoken user input and content being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 52 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a position of the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 53 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 54 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a semantic relationship between the first spoken user input an application being run by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input device based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 55 (depends on 36)

The non-transitory computer readable storage medium of claim 36, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.

### Claim 56 (depends on 37)

The system of claim 37, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input excludes identifying a physical or virtual button input received prior to receiving the first spoken user input.

### Claim 57 (depends on 37)

The system of claim 37, wherein determining whether to respond to the first spoken user input based on contextual information associated with the first spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the first spoken user input.

### Claim 58 (depends on 37)

The system of claim 37, wherein the contextual information comprises an elapsed time between receiving the first spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 59 (depends on 37)

The system of claim 37, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the first spoken user input.

### Claim 60 (depends on 37)

The system of claim 37, wherein the contextual information comprises an orientation of the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 61 (depends on 37)

The system of claim 37, wherein the contextual information comprises an orientation between the user and the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and
decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 62 (depends on 37)

The system of claim 37, wherein the contextual information comprises an indication of whether the first spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the first spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the first spoken user input was not recognized by the automatic speech recognizer.

### Claim 63 (depends on 37)

The system of claim 37, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 64 (depends on 37)

The system of claim 37, wherein the contextual information comprises an identification of a speaker of the first spoken user input.

### Claim 65 (depends on 37)

The system of claim 37, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the first spoken user input was received.

### Claim 66 (depends on 37)

The system of claim 37, wherein the contextual information comprises an indication of whether the electronic device is being held when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the first spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the first spoken user input was received.

### Claim 67 (depends on 37)

The system of claim 37, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 68 (depends on 37)

The system of claim 37, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the first spoken user input was received.

### Claim 69 (depends on 37)

The system of claim 37, wherein the contextual information comprises a semantic relationship between the first spoken user input and content being displayed by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 70 (depends on 37)

The system of claim 37, wherein the contextual information comprises a position of the user when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 71 (depends on 37)

The system of claim 37, wherein the contextual information comprises a semantic relationship between the first spoken user input and the previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 72 (depends on 37)

The system of claim 37, wherein the contextual information comprises a semantic relationship between the first spoken user input an application being run by the electronic device when the first spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input device based on contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 73 (depends on 38)

The system of claim 38, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the audible response to the first spoken user input based on contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.