# US 2016/0260436 A1 · Virtual assistant activation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0260436 A1
- Document ID: US20160260436A1
- Status: Published application
- Kind: A1
- Application number: 14/841,449
- Filed: 2015-08-31
- Published: 2016-09-08
- Earliest claimed: 2015-03-08
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Stephen O. LEMAY; Brandon J. NEWENDORP; Jonathan R. DASCOLA
- Web page: https://patentgrep.com/patent/US20160260436A1

## CPC

- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G10L 15/28: Constructional details of speech recognition systems
- H04L 12/1827: {Network arrangements for conference optimisation or adaptation}
- H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 2015/088: {Word spotting}
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- H04L 51/10: Multimedia information
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

At an electronic device with a display, a microphone, and an input device: while the display is on, receiving user input via the input device, the user input meeting a predetermined condition; in accordance with receiving the user input meeting the predetermined condition, sampling audio input received via the microphone; determining whether the audio input comprises a spoken trigger; and in accordance with a determination that audio input comprises the spoken trigger, triggering a virtual assistant session.

## Claims

### Claim 1

A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which when executed by an electronic device with a display, a microphone, and an input device, cause the electronic device to:
while the display is on, receive user input via the input device, the user input meeting a predetermined condition;
in accordance with receiving the user input meeting the predetermined condition, sample audio input received via the microphone;
determine whether the audio input comprises a spoken trigger; and
in accordance with a determination that audio input comprises the spoken trigger, trigger a virtual assistant session.

### Claim 2 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the predetermined condition is based on an amount of lifting of the electronic device during a time interval, the amount of lifting determined from an accelerometer of the electronic device.

### Claim 3 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the predetermined condition is based on smoothness of a lifting of the electronic device during a time interval, the smoothness of the lifting determined from an accelerometer of the electronic device.

### Claim 4 (depends on 2)

The non-transitory computer readable storage medium of claim 2, wherein the predetermined condition comprises a minimum period of dwell time in a position in accordance with the lifting of the electronic device.

### Claim 5 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the predetermined condition comprises detecting activation of a button of the electronic device.

### Claim 6 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the predetermined condition comprises detecting touch input on a touch-sensitive surface of the electronic device.

### Claim 7 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
determine whether the display is on; and
wherein the sampling of audio input occurs in accordance with a determination that the display is on.

### Claim 8 (depends on 7)

The non-transitory computer readable storage medium of claim 7, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
in accordance with determining whether the display is on, determine whether the backlight of the display is on.

### Claim 9 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
cease the sampling of audio input after the sampling of audio input has occurred for a predetermined duration of time.

### Claim 10 (depends on 9)

The non-transitory computer readable storage medium of claim 9, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
in accordance with ceasing to sample audio input, provide a haptic output.

### Claim 11 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the user input meeting a predetermined condition is a first user input meeting a first predetermined condition, and wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
while sampling audio input in accordance with receiving the first user input, receive a second user input meeting a second predetermined condition; and
in accordance with receiving the second user input meeting the second predetermined condition, cease the sampling of audio input.

### Claim 12 (depends on 11)

The non-transitory computer readable storage medium of claim 11, wherein the second predetermined condition is based on an amount of lowering of the electronic device during a time interval, the amount of lowering determined from an accelerometer of the electronic device.

### Claim 13 (depends on 11)

The non-transitory computer readable storage medium of claim 11, wherein the first predetermined condition is based on smoothness of a lowering of the electronic device during a time interval, the smoothness of the lowering determined from an accelerometer of the electronic device.

### Claim 14 (depends on 11)

The non-transitory computer readable storage medium of claim 11, wherein the second predetermined condition comprises detecting activation of a button of the electronic device.

### Claim 15 (depends on 11)

The non-transitory computer readable storage medium of claim 11, wherein the second predetermined condition comprises detecting touch input on a touch-sensitive surface of the electronic device.

### Claim 16 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
while sampling audio input in accordance with receiving the user input meeting the predetermined condition, determine whether the display is off; and
in accordance with a determination that the display is off, cease the sampling of audio input.

### Claim 17 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
while sampling audio input in accordance with receiving the user input meeting the predetermined condition, identify an audio endpoint in the audio input; and
in response to identifying the audio endpoint, cease the sampling of audio input.

### Claim 18 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
in accordance with triggering the virtual assistant session, prompt a user for spoken input.

### Claim 19 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the audio input comprises the spoken trigger and additional input, and wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
in accordance with triggering the virtual assistant session:
determine a user intent based on at least the additional input in the audio input, and
execute a task associated with the user intent.

### Claim 20 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the non-transitory computer-readable storage medium further comprises instructions, which when executed by the electronic device, cause the electronic device to:
in accordance with a determination that the electronic device is operating in a predetermined mode, forgo sampling of audio input, even after receiving the user input meeting the predetermined condition while the display is on,
wherein operation in the predetermined mode comprises muting audio output of the electronic device.

### Claim 21 (depends on 1)

The non-transitory computer readable storage medium of claim 1, wherein the spoken trigger comprises comprise a predetermined phrase.

### Claim 22

A method, comprising:
at an electronic device with a display, a microphone, and an input device:
while the display is on, receiving user input via the input device, the user input meeting a predetermined condition;
in accordance with receiving the user input meeting the predetermined condition, sampling audio input received via the microphone;
determining whether the audio input comprises a spoken trigger; and
in accordance with a determination that audio input comprises the spoken trigger, triggering a virtual assistant session.

### Claim 23

An electronic device comprising:
a display;
one or more processors;
a memory; and
one or more programs, wherein the one or more programs are stored in memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
while the display is on, receiving user input via an input device, the user input meeting a predetermined condition;
in accordance with receiving the user input meeting the predetermined condition, sampling audio input received via a microphone;
determining whether the audio input comprises a spoken trigger; and
in accordance with a determination that audio input comprises the spoken trigger, triggering a virtual assistant session.
