# US 12,181,847 B1 · Activity-based device recommendations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,181,847 B1
- Document ID: US12181847B1
- Status: Granted
- Kind: B1
- Application number: 16/832,385
- Filed: 2020-03-27
- Granted: 2024-12-31
- Earliest claimed: 2020-03-27
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Charles Edwin Ashton Brett; William Evan Welbourne; Hongyang Wang; Akshay Kumar; George Strajan
- Web page: https://patentgrep.com/patent/US12181847B1

## CPC

- G05B 19/042: using digital processors (G05B19/05 takes precedence)
- G06F 1/3296: by lowering the supply or operating voltage
- G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback}
- G06N 20/00: Machine learning
- G06V 10/60: relating to illumination properties, e.g. using a reflectance or lighting model
- G08B 13/00: Burglar, theft or intruder alarms
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G05B 2219/2642: Domotique, domestic, home control, automation, smart house
- G08B 21/0407: {based on behaviour analysis}
- G10L 2015/223: {Execution procedure of a spoken command}
- G16H 40/67: for remote operation
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and methods for activity-based device recommendations are disclosed. For example, historical usage data associated with a device may indicate that the device is likely to be associated with a given state at a given time. When the device is not in the anticipated state, a recommendation to transition the device state, for example, may be sent. Additionally, a determination of the activity state associated with the device, such as an active state, an asleep state, and/or an away state may be utilized to determine the recommendation to surface, to determine whether to send a recommendation, and when and/or how to send the recommendation.

## Claims

### Claim 1

A method, comprising:
storing first data associating a user profile with a plurality of devices;
receiving, from a first device of the plurality of devices, second data indicating that a user device associated with the user profile is present in relation to the first device;
determining that the first device is disposed within a room having a room identifier and the user device is disposed within the room, wherein the room identifier for the room differs from other room identifiers for other rooms associated with the user profile;
determining, based at least in part on third data indicating states of a second device of the plurality of devices, that a state of the second device disposed within the room remains constant for a period of time that is at least a threshold period of time, the state predetermined to correspond to device inactivity;
determining, utilizing machine learning, fourth data indicating:
a first device state transition from a first operational state to a second operational state indicating historical cessation of an asleep state for the room identifier, the asleep state associated with a person sleeping; and
a second device state transition from the second operational state to the first operational state indicating historical initiation of the asleep state for the room identifier, wherein the first device state transition and the second device state transition differ for at least one of the other room identifiers for the other rooms;
determining, during the period of time, an absence of a first acoustic event predefined as being associated with an awake state within the room;
determining, during the period of time, that a second acoustic event predefined as being associated with the asleep state is detected within the room;
associating the room identifier of the room with the asleep state based at least in part on:
the user device being present in relation to the first device;
the state of the second device remaining constant for the period of time;
the second device performing the second device state transition;
determining the absence of the first acoustic event; and
determining the second acoustic event has been detected; and
causing, based at least in part on associating the room identifier with the asleep state, one or more smart home applications associated with the user profile to perform at least one of first notification actions or first recommendation actions associated with the room that air predefined to be performed while the user profile is associated with the asleep state, the at least one of the first notification actions or the first recommendation actions differing from other notification actions and other recommendation actions performed by the one or more smart home applications when associated with the awake state.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining that receipt of audio data including user speech from the first device during the period of time is absent; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on determining that receipt of the audio data during the period of time is absent.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
sending, to the first device, request data for an indication of acoustic events detected during the period of time;
receiving, from the first device, fifth data indicating that the first acoustic event was undetected; and
wherein determining the absence of the first acoustic event is based at least in part on the fifth data.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving fifth data indicating that a security system associated with the user profile is in a home state, the home state disregarding motion detected by a motion sensor of the security system; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the fifth data.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving fifth data indicating that a force sensor of a third device, associated with a bed disposed in an environment, has detected a force value that satisfies a threshold force value, the threshold force value indicating that a user is laying on the bed; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the fifth data.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
determining to enable a routine associated with inactivity in an environment; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on determining to enable the routine.

### Claim 7

A system, comprising:
one or more processors; and
non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
storing first data associating a user profile with a plurality of devices;
receiving, from a first device of the plurality of devices, second data indicating that a user device associated with the user profile is present in relation to the first device;
determining that the first device is disposed within a room having a room identifier and the user device is disposed within the room, wherein the room identifier for the room differs from other room identifiers for other rooms associated with the user profile;
determining, based at least in part on third data indicating states of a second device of the plurality of devices, that a state of the second device disposed within the room remains constant for a period of time that is at least a threshold period of time, the state predetermined to correspond to device inactivity;
determining, utilizing machine learning, fourth data indicating:
a first device state transition from a first operational state to a second operational state indicating historical cessation of an asleep state for the room identifier, the asleep state associated with a person sleeping; and
a second device state transition from the second operational state to the first operational state indicating historical initiation of the asleep state for the room identifier, wherein the first device state transition and the second device state transition differ for at least one of the other room identifiers for the other rooms;
determining, during the period of time, an absence of a first acoustic event predefined as being associated with an awake state within the room;
determining, during the period of time, that a second acoustic event predefined as being associated with the asleep state is detected within the room;
associating the room identifier of the room with an asleep state based at least in part on:
the user device being present in relation to the first device;
the state of the second device remaining constant for the period of time;
the second device performing the second device state transition;
determining the absence of the first acoustic event; and
determining the second acoustic event has been detected; and
causing, based at least in part on associating the room identifier with the asleep state, one or more smart home applications associated with the user profile to perform at least one of first notification actions or first recommendation actions within the room predefined to be performed while the user profile is associated with the asleep state, the at least one of first notification actions or the first recommendation actions differing from other notification actions and other recommendation actions performed by the one or more smart home applications when associated with the awake state.

### Claim 8 (depends on 7)

The system of claim 7, the operations further comprising:
determining that receipt of audio data including user speech from the first device during the period of time is absent; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on determining that receipt of the audio data during the period of time is absent.

### Claim 9 (depends on 7)

The system of claim 7, the operations further comprising:
sending, to the first device, request data for an indication of acoustic events detected during the period of time;
receiving, from the first device, fifth data indicating that the first acoustic event was undetected; and
wherein determining the absence of the first acoustic event is based at least in part on the fifth data.

### Claim 10 (depends on 7)

The system of claim 7, the operations further comprising:
receiving, while the user profile is associated with the asleep state, an indication that the second device has transitioned from the first operational state to the second operational state;
determining, based at least in part on the indication, that a command to transition the second device was sent based at least in part on user input; and
associating, based at least in part on determining that the command was sent in response to the user input, the room identifier with an active state.

### Claim 11 (depends on 7)

The system of claim 7, the operations further comprising:
receiving, from the first device, fifth data indicating that an ambient light value determined by an ambient light sensor in an environment satisfies a threshold ambient light value; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the fifth data.

### Claim 12 (depends on 7)

The system of claim 7, the operations further comprising:
receiving fifth data indicating that a security system associated with the user profile is in a home state, the home state disregarding motion detected by a motion sensor of the security system; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the fifth data.

### Claim 13 (depends on 7)

The system of claim 7, the operations further comprising:
receiving fifth data indicating that a force sensor of a third device, associated with a bed disposed in an environment, has detected a force value that satisfies a threshold force value, the threshold force value indicating that a user is laying on the bed; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the fifth data.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
receiving audio data from the first device during the period of time;
determining that a portion of the audio data corresponds at least in part to user sound;
determining that the portion of the audio data lacks speech input; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the portion of the audio data lacking the speech input.

### Claim 15 (depends on 1)

The method of claim 1, further comprising:
querying, based at least in part on the state of the second device remaining constant for the period of time, the plurality of devices for presence data indicating user presence;
receiving response data indicating user presence was unidentified by the plurality of devices; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the response data.

### Claim 16 (depends on 7)

The system of claim 7, the operations further comprising:
receiving audio data from the first device during the period of time;
determining that a portion of the audio data corresponds at least in part to user sound;
determining that the portion of the audio data lacks speech input; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the portion of the audio data lacking the speech input.

### Claim 17 (depends on 7)

The system of claim 7, the operations further comprising:
querying, based at least in part on the state of the second device remaining constant for the period of time, the plurality of devices for presence data indicating user presence;
receiving response data indicating user presence was unidentified by the plurality of devices; and
wherein associating the room identifier with the asleep state comprises associating the room identifier with the asleep state based at least in part on the response data.

### Claim 18 (depends on 1)

The method of claim 1, further comprising:
querying a model to predict future activity states associated with the second device; and
receiving predicted future activity states from the model in response to the querying, and wherein associating the room identifier with the asleep state is based at least in part on the predicted future activity states.

### Claim 19 (depends on 1)

The method of claim 1, further comprising:
identifying a content injection framework associated with the asleep state; and
determining when to cause the one or more smart home applications to perform the at least one of first notification actions or the first recommendation actions based at least in part on the content injection framework.

### Claim 20 (depends on 1)

The method of claim 1, wherein determining the fourth data is performed utilizing a machine learning model trained to determine the first device state transition and the second device state transition.