# US 10,770,073 B2 · Reducing the need for manual start/end-pointing and trigger phrases

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,770,073 B2
- Document ID: US10770073B2
- Status: Granted
- Kind: B2
- Application number: 16/530,708
- Filed: 2019-08-02
- Published: 2020-03-26
- Granted: 2020-09-08
- Earliest claimed: 2014-05-30
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Philippe P. Piernot; Justin Binder
- Web page: https://patentgrep.com/patent/US10770073B2

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)}
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- H04W 4/025: {using location based information parameters}
- G06F 2203/0381: Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/1822: {Parsing for meaning understanding}
- G10L 17/00: Speaker identification or verification techniques
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/227: {of the speaker; Human-factor methodology}
- G10L 2015/228: {of application context}
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Systems and processes for selectively processing and responding to a spoken user input are provided. In one example, audio input containing a spoken user input can be received at a user device. The spoken user input can be identified from the audio input by identifying start and end-points of the spoken user input. It can be determined whether or not the spoken user input was intended for a virtual assistant based on contextual information. The determination can be made using a rule-based system or a probabilistic system. If it is determined that the spoken user input was intended for the virtual assistant, the spoken user input can be processed and an appropriate response can be generated. If it is instead determined that the spoken user input was not intended for the virtual assistant, the spoken user input can be ignored and/or no response can be generated.

## Claims

### Claim 1

A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by one or more processors of an electronic device, cause the device to:
receive, from a user, a spoken user input comprising a user request;
determine whether to respond to the spoken user input based on contextual information associated with the spoken user input, wherein the contextual information comprises a direction of a user's gaze when the spoken user input was received, wherein the determining comprises:
calculating a likelihood score that a virtual assistant should provide a response to the spoken user input based on the contextual information associated with the spoken user input, wherein the response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the spoken user input was received;
in response to a determination to respond to the spoken user input:
generate the response to the spoken user input; and
output the response.

### Claim 2 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the one or more programs further comprise instructions, which when executed by the one or more processors, cause the device to:
in response to a determination not to respond to the spoken user input:
forgo generating the response to the spoken user input; and
monitor for a second spoken user input.

### Claim 3 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying one or more predetermined words at a start of the spoken user input.

### Claim 4 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying a physical or virtual button input received prior to receiving the spoken user input.

### Claim 5 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the spoken user input.

### Claim 6 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises an elapsed time between receiving the spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 7 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the spoken user input.

### Claim 8 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises an orientation of the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 9 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises an orientation between the user and the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and
decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 10 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises an indication of whether the spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the spoken user input was not recognized by the automatic speech recognizer.

### Claim 11 (depends on 1)

The non-transitory computer-readable storage medium of claim 1, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 12 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises an identification of a speaker of the spoken user input.

### Claim 13 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the spoken user input was received.

### Claim 14 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises an indication of whether the electronic device is being held when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the spoken user input was received.

### Claim 15 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 16 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the spoken user input was received.

### Claim 17 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a semantic relationship between the spoken user input and content being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 18 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a position of the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 19 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 20 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a semantic relationship between the spoken user input and an application being run by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input device based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 21 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.

### Claim 22

A method for operating a virtual assistant, the method comprising:
at an electronic device with one or more processors and memory:
receiving, from a user, a spoken user input comprising a user request;
determining whether to respond to the spoken user input based on contextual information associated with the spoken user input, wherein the contextual information comprises a direction of a user's gaze when the spoken user input was received, wherein the determining comprises:
calculating a likelihood score that the virtual assistant should provide a response to the spoken user input based on the contextual information associated with the spoken user input, wherein the response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the spoken user input was received;
in response to a determination to respond to the spoken user input:
generating the response to the spoken user input; and
outputting the response.

### Claim 23

A system for operating a virtual assistant, the system comprising:
one or more processors;
memory; and
one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
receiving, from a user, a spoken user input comprising a user request;
determining whether to respond to the spoken user input based on contextual information associated with the spoken user input, wherein the contextual information comprises a direction of a user's gaze when the spoken user input was received, wherein the determining comprises:
calculating a likelihood score that the virtual assistant should provide a response to the spoken user input based on the contextual information associated with the spoken user input, wherein the response at least partially satisfies the user request;
increasing the likelihood score in response to the direction of the user's gaze being pointed at the electronic device when the spoken user input was received; and
decreasing the likelihood score in response to the direction of the user's gaze being pointed away from the electronic device when the spoken user input was received;
in response to a determination to respond to the spoken user input:
generating the response to the spoken user input; and
outputting the response.

### Claim 24 (depends on 22)

The method of claim 22, further comprising:
in response to a determination not to respond to the spoken user input:
forgoing generating the response to the spoken user input; and
monitoring for a second spoken user input.

### Claim 25 (depends on 22)

The method of claim 22, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying one or more predetermined words at a start of the spoken user input.

### Claim 26 (depends on 22)

The method of claim 22, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying a physical or virtual button input received prior to receiving the spoken user input.

### Claim 27 (depends on 22)

The method of claim 22, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the spoken user input.

### Claim 28 (depends on 22)

The method of claim 22, wherein the contextual information comprises an elapsed time between receiving the spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 29 (depends on 22)

The method of claim 22, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the spoken user input.

### Claim 30 (depends on 22)

The method of claim 22, wherein the contextual information comprises an orientation of the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 31 (depends on 22)

The method of claim 22, wherein the contextual information comprises an orientation between the user and the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and
decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 32 (depends on 22)

The method of claim 22, wherein the contextual information comprises an indication of whether the spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the spoken user input was not recognized by the automatic speech recognizer.

### Claim 33 (depends on 22)

The method of claim 22, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 34 (depends on 22)

The method of claim 22, wherein the contextual information comprises an identification of a speaker of the spoken user input.

### Claim 35 (depends on 22)

The method of claim 22, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the spoken user input was received.

### Claim 36 (depends on 22)

The method of claim 22, wherein the contextual information comprises an indication of whether the electronic device is being held when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the spoken user input was received.

### Claim 37 (depends on 22)

The method of claim 22, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 38 (depends on 22)

The method of claim 22, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the spoken user input was received.

### Claim 39 (depends on 22)

The method of claim 22, wherein the contextual information comprises a semantic relationship between the spoken user input and content being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 40 (depends on 22)

The method of claim 22, wherein the contextual information comprises a position of the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 41 (depends on 22)

The method of claim 22, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 42 (depends on 22)

The method of claim 22, wherein the contextual information comprises a semantic relationship between the spoken user input and an application being run by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input device based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 43 (depends on 22)

The method of claim 22, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.

### Claim 44 (depends on 23)

The system of claim 23, wherein the one or more programs further include instructions for:
in response to a determination not to respond to the spoken user input:
forgoing generating the response to the spoken user input; and monitoring for a second spoken user input.

### Claim 45 (depends on 23)

The system of claim 23, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying one or more predetermined words at a start of the spoken user input.

### Claim 46 (depends on 23)

The system of claim 23, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input excludes identifying a physical or virtual button input received prior to receiving the spoken user input.

### Claim 47 (depends on 23)

The system of claim 23, wherein determining whether to respond to the spoken user input based on contextual information associated with the spoken user input comprises:
evaluating one or more conditional rules that depend on the contextual information associated with the spoken user input.

### Claim 48 (depends on 23)

The system of claim 23, wherein the contextual information comprises an elapsed time between receiving the spoken user input and a previous user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to a value of the elapsed time being greater than a threshold duration; and
increasing the likelihood score in response to the value of the elapsed time being less than the threshold duration.

### Claim 49 (depends on 23)

The system of claim 23, wherein the contextual information comprises a previous spoken user input, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to detecting a match between the previous spoken user input and the spoken user input.

### Claim 50 (depends on 23)

The system of claim 23, wherein the contextual information comprises an orientation of the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to spoken user input based on the contextual information comprises:
decreasing the likelihood score in response to the orientation of the device being face down or upside down; and
increasing the likelihood score in response to the orientation of the device being face up or upright.

### Claim 51 (depends on 23)

The system of claim 23, wherein the contextual information comprises an orientation between the user and the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the orientation being one in which a display of the electronic device is oriented towards the user; and decreasing the likelihood score in response to the orientation being one in which the display of the electronic device is oriented away from the user.

### Claim 52 (depends on 23)

The system of claim 23, wherein the contextual information comprises an indication of whether the spoken user input was recognized by an automatic speech recognizer, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the spoken user input was recognized by the automatic speech recognizer; and
decreasing the likelihood score in response to the indication indicating that the spoken user input was not recognized by the automatic speech recognizer.

### Claim 53 (depends on 23)

The system of claim 23, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous spoken user input, and wherein determining the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a spoken user input semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the spoken user input semantic threshold value.

### Claim 54 (depends on 23)

The system of claim 23, wherein the contextual information comprises an identification of a speaker of the spoken user input.

### Claim 55 (depends on 23)

The system of claim 23, wherein the contextual information comprises an indication of whether the electronic device was outputting information to the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was outputting information to the user when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not outputting information to the user when the spoken user input was received.

### Claim 56 (depends on 23)

The system of claim 23, wherein the contextual information comprises an indication of whether the electronic device is being held when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the indication indicating that the electronic device was being held when the spoken user input was received; and
decreasing the likelihood score in response to the indication indicating that the electronic device was not being held when the spoken user input was received.

### Claim 57 (depends on 23)

The system of claim 23, wherein the contextual information comprises a previous action performed by the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the previous action performed by the electronic device being one of a set of predetermined actions; and
decreasing the likelihood score in response to the previous action performed by the electronic device not being one of the set of predetermined actions.

### Claim 58 (depends on 23)

The system of claim 23, wherein the contextual information comprises an indication of whether content was being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises increasing the likelihood score in response to the indication indicating that content was being displayed by the electronic device when the spoken user input was received.

### Claim 59 (depends on 23)

The system of claim 23, wherein the contextual information comprises a semantic relationship between the spoken user input and content being displayed by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a content semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the content semantic threshold value.

### Claim 60 (depends on 23)

The system of claim 23, wherein the contextual information comprises a position of the user when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the position of the user being one of a predetermined set of positions; and
decreasing the likelihood score in response to the position of the user not being one of the predetermined set of positions.

### Claim 61 (depends on 23)

The system of claim 23, wherein the contextual information comprises a semantic relationship between the spoken user input and a previous output of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than a previous output semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the previous output semantic threshold value.

### Claim 62 (depends on 23)

The system of claim 23, wherein the contextual information comprises a semantic relationship between the spoken user input and an application being run by the electronic device when the spoken user input was received, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input device based on the contextual information comprises:
increasing the likelihood score in response to a value of the semantic relationship being greater than an application semantic threshold value; and
decreasing the likelihood score in response to the value of the semantic relationship being less than the application semantic threshold value.

### Claim 63 (depends on 23)

The system of claim 23, wherein the contextual information comprises a movement of the electronic device, and wherein calculating the likelihood score that the virtual assistant should provide the response to the spoken user input based on the contextual information comprises:
increasing the likelihood score in response to the movement being one of a predetermined set of movements; and
decreasing the likelihood score in response to the movement not being one of the predetermined set of movements.