US 10,121,494 B1 · User presence detection

Abstract

A speech-capture device can capture audio data during wakeword monitoring and use the audio data to determine if a user is present nearby the device, even if no wakeword is spoken. Audio such as speech, human originating sounds (e.g., coughing, sneezing), or other human related noises (e.g., footsteps, doors closing) can be used to detect audio. Audio frames are individually scored as to whether a human presence is detected in the particular audio frames. The scores are then smoothed relative to nearby frames to create a decision for a particular frame. Presence information can then be sent according to a periodic schedule to a remote device to create a presence “heartbeat” that regularly identifies whether a user is detected proximate to a speech-capture device.

Claims31

Dependent claims

A computer-implemented method comprising:

receiving first audio data corresponding to first audio in an environment of a device;
processing the first audio data using a deep neural network (DNN) model to determine a first score corresponding to a first audio frame of the first audio data, the first score corresponding to a likelihood that human activity was present in the environment;
determining that the first score is above a score threshold;
sending, at a first time, a first indicator of human presence to a remote device;
receiving second audio data corresponding to second audio in the environment, the second audio occurring after the first audio;
processing the second audio data using the DNN model to determine a second score corresponding to a second audio frame of the second audio data, the second score corresponding to a likelihood that human activity was present in the environment;
determining that the second score is above the score threshold;
and
sending, at a second time, a second indicator of human presence to the remote device, the second time being a preconfigured time interval after the first time.

6 more elements

A device comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to perform a set of actions to configure the device to:
receive first audio data corresponding to first audio captured by at least one microphone;
process the first audio data to determine a first score corresponding to a likelihood that human activity was present in an environment of the at least one microphone;
determine that the first score is above a score threshold;
determine a first indicator of user presence based at least in part on the first score;
send, at a first time, the first indicator of user presence to a remote device;
receive second audio data corresponding to second audio captured by the at least one microphone;
process the second audio data to determine that user presence is represented in the second audio data;
and
send, at a second time, a second indicator of user presence to the remote device, wherein the second time is a preconfigured time interval after the first time.

9 more elements

A device, comprising:

at least one processor;
and
memory including instructions operable to be executed by the at least one processor to perform a set of actions to configure the device to:
receive first audio data corresponding to first audio captured by at least one microphone,
process the first audio data to determine that user presence is represented in the first audio data,
send, at a first time, a first indicator of user presence to a remote device,
receive second audio data corresponding to second audio captured by the at least one microphone,
process the second audio data to determine that user presence is represented in the second audio data,
send, at a second time, a second indicator of user presence to the remote device, wherein the second time is a preconfigured time interval after the first time, and
send at least a portion of the first audio data to the remote device for speech processing.

7 more elements

Dependencies are inferred from the claim text.

User presence detection

Life of this patent

User presence detection

Life of this patent

Abstract

Claims31