# US 2020/0404077 A1 · Wearable device for controlling endpoint devices

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404077 A1
- Document ID: US20200404077A1
- Status: Published application
- Kind: A1
- Application number: 16/450,180
- Filed: 2019-06-24
- Published: 2020-12-24
- Earliest claimed: 2019-06-24
- Assignees: Amazon Technologies, Inc.
- Applicants: Amazon Technologies, Inc.
- Inventors: Tao Jiang; Hans Edward Birch-Jensen
- Web page: https://patentgrep.com/patent/US20200404077A1

## CPC

- H04L 67/34: {involving the movement of software or configuration parameters (network booting or remote initial program loading [RIPL] G06F9/4416)}
- G06F 1/163: {Wearable computers, e.g. on a belt}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 16/9035: Filtering based on additional data, e.g. user or group profiles
- G06N 20/00: Machine learning
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- H04W 4/38: for collecting sensor information
- H04W 12/33: using wearable devices, e.g. using a smartwatch or smart-glasses
- H04W 12/63: Location-dependent; Proximity-dependent
- H04W 12/68: Gesture-dependent or behaviour-dependent
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

Technologies for a wearable device are described. One wearable device includes a radio and a processor, the processor measures sensor data (e.g., a first angle between the wearable device and a first wireless endpoint device and a second angle between the wearable device and a second wireless endpoint device) and motion data indicative of motion of the wearable device over a first duration of time. The wearable device also measure signal strength values for communications with the respective devices. The wearable device predicts, using a first trained model, a position of the wearable device and to which target device the wearable device is directed. The wearable device predicts, using a second trained mode, a gesture made by the wearable device. The wearable device sends a message, corresponding to the gesture, to the target device to control the target device.

## Claims

### Claim 1

A wearable electronic device comprising:
a user interface device;
a single radio with a multi-mode physical (PHY) layer;
a motion tracking sensor comprising a multi-axis gyroscope and an accelerometer;
a processor coupled to the user interface device, the single radio, and the motion tracking sensor, wherein the processor:
receives, via the user interface device, a request that causes the processor to enter a first mode in which the processor establishes a secure wireless connection with a wireless access point device and downloads a first trained model for gesture recognition, a second trained model for target device predictions, and a device database, wherein the device database comprises a set of device identifiers that represent a plurality of devices registered to a user account and a controlling profile for each device of the plurality of devices;
discovers, in a second mode, a first wireless light bulb and a second wireless light bulb;
determines first angle data between the wearable electronic device and the first wireless light bulb and second angle data between the wearable electronic device and the second wireless light bulb;
determines, in the second mode, a first receive signal strength indicator (RSSI) value and a second RSSI value, wherein the first RSSI value is measured at the single radio and is indicative of a first distance between the wearable electronic device and the first wireless light bulb and the second RSSI value is measured at the single radio and is indicative of a second distance between the wearable electronic device and the second wireless light bulb;
determines, using the first trained model and acceleration data received from the motion tracking sensor, a first gesture performed by a user of the wearable electronic device, wherein the first gesture is associated with a first command;
determines, using the second trained model, the second angle data, and the second RSSI value, that the first gesture was associated with the second wireless light bulb; and
sends, to the second wireless light bulb, data associated with the command.

### Claim 2 (depends on 1)

The wearable electronic device of claim 1, wherein the processor further:
before the data associated with the command is sent to the second wireless light bulb, verifies that the second wireless light bulb is among the devices registered to the user account in the device database; and
before the data associated with the command is sent to the second wireless light bulb, verifies that the second RSSI value exceeds a RSSI threshold specified in a rule stored in a policy database.

### Claim 3 (depends on 1)

The wearable electronic device of claim 1, wherein the processor further:
detects, during an idle mode of the wearable electronic device, a motion of the wearable electronic device using the acceleration data from the motion tracking sensor;
transitions from the idle mode to the second mode;
initiates, during the second mode, a first discovery process to discover any wireless light bulbs using a first protocol via the single radio; and
initiates, during the second mode and after the first discovery process, a second discovery process to discover any wireless light bulbs using a second protocol via the single radio.

### Claim 4 (depends on 1)

The wearable electronic device of claim 1, wherein the processor further:
before downloading the second trained model from the wireless access point device,
determines first historical angle data between the wearable electronic device and the first wireless light bulb and second historical angle data between the wearable electronic device and the second wireless light bulb
determines first historical RSSI values, each corresponding to a distance between the wearable electronic device and the first wireless light bulb, and second historical RSSI values, each correspond ding to a distance between the wearable electronic device and the second wireless light bulb; and
sends the first historical angle data, the first historical RSSI values, the second historical angle data, and the second historical RSSI values to a remote server; and receives the second trained model from the remote server.

### Claim 5

A method comprising:
discovering, by a processor of a wearable device, a first device and a second device via a radio of the wearable device;
receiving, by the processor, sensor data from a motion sensor of the wearable device, wherein the sensor data comprises (i) first angle data between the wearable device and the first device, (ii) second angle data between the wearable device and the second device and (iii) motion data indicative of motion of the wearable device over a first duration of time;
determining, by the processor, a first radio frequency (RF) signal strength value of a first wireless connection between the first device and the wearable device;
determining, by the processor a second RF signal strength value of a second wireless connection between the second device and the wearable device;
determining, by the processor, using the first angle data, the second angle data, the first RF signal strength value, and the second RF signal strength value, iv) a position of the wearable device and v) that the position of the wearable device indicates selection of the second device;
determining, by the processor using the sensor data, a gesture associated with the motion of the wearable device; and
sending, by the processor, a message corresponding to the gesture to the second device.

### Claim 6 (depends on 5)

The method of claim 5, wherein determining iv) the position of the wearable device and v) that the position of the wearable device indicates selection of the second device, comprises determining iv) the position of the wearable device and v) that the position of the wearable device indicates selection of the second device using a first trained model, wherein the first angle data, the second angle data, the first RF signal strength value, and the second RF signal strength value are inputs into the first trained model.

### Claim 7 (depends on 5)

The method of claim 5, further comprising:
receiving, by the processor via a user interface device of the wearable device, user input;
establishing, in response to the user input and by the processor via the radio, a wireless connection with a wireless access point device, wherein the wireless access point device is communicatively coupled to the first and the second devices; and
receiving a first trained model, a second trained model, and a device database via the wireless access point device, wherein the device database comprises (vi) a first device identifier associated with the first device, (vii) a second device identifier associated with the second device and (viii) profile information for each of the first and the second devices and wherein the first and the second devices are associated with a user account.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
before receiving the first trained model via the wireless access point device,
determining first historical angle data between the wearable device and the first device and second historical angle data between the wearable device and the second device;
determining ix) first historical RSSI values, each corresponding to a distance between the wearable device and the first device, and x) second historical RSSI values, each corresponding to a distance between the wearable device and the second device; and
sending the first historical angle data, the first historical RSSI values, the second historical angle data, and the second historical RSSI values to a remote server; and
receiving the second trained model from the remote server.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
sending, by the processor, the sensor data, the first RF signal strength value, and the second RF signal strength value to a remote server via the radio; and
receiving, by the processor, the first trained model from the remote server.

### Claim 10 (depends on 7)

The method of claim 7, further comprising:
verifying that the second device is among devices registered to the user account in the device database before the first message is sent to the second device; and
verifying that the second RF signal strength value exceeds a threshold specified in a rule stored in a policy database before the message is sent to the second device.

### Claim 11 (depends on 5)

The method of claim 5, further comprising:
detecting, by the processor during an idle mode of the wearable device before the discovering the first device, a motion of the wearable device using the sensor data from the motion sensor;
transitioning, by the processor, from the idle mode to a discovery mode;
initiating, by the processor during the discovery mode, a first discovery process with a first protocol to discover any devices using the first protocol; and
initiating, by the processor during the discovery mode and after the first discovery process, a second discovery process with a second protocol to discover any devices using a second protocol.

### Claim 12 (depends on 5)

The method of claim 5, further comprising:
receiving, by a multi-protocol stack of the processor, a first packet from a multi-mode physical (PHY) layer of the radio, wherein the radio is a single radio; and
sending, by the multi-protocol stack, a second packet to the multi-mode PHY layer.

### Claim 13 (depends on 5)

The method of claim 5, wherein sending the message comprises:
receiving, by the processor via a wireless access point, a device database comprising (vi) a first device identifier associated with the first device, (vii) a second device identifier associated with the second device, (viii) a first link key and a first network key associated with the first device, and ix) a second link key and a second network key associated with the second device; and
generating, by the processor, the message using the second link key and the second network key, wherein formatting the message comprises encrypting a command corresponding to the gesture with the second link key and the second network key.

### Claim 14 (depends on 5)

The method of claim 5, wherein sending the message comprises:
retrieving, by the processor, a controlling profile associated with the second device in a device database;
retrieving, by the processor, an link key and a network key associated with the second device in a key store; and
generating, by a multi-protocol stack of the processor, the message using the network key and the link key.

### Claim 15

A method comprising:
discovering, by a processor of a wearable device, a first device, a second device, and a third device via a radio of the wearable device;
receiving, by the processor, motion data from a motion sensor of the wearable device during movement of the wearable device in a pattern that is directed towards the first device and directed away from the first device;
determining, by the processor, first proximity data between the wearable device and the first device at discrete points in time during the movement of the wearable device in the pattern;
determining, by the processor, second proximity data between the wearable device and the second device at the same discrete points in time during the movement of the wearable device in the pattern;
determining, by the processor, third proximity data between the wearable device and the third device at the same discrete points in time during the movement of the wearable device in the pattern;
determining, by the processor, which of the first, second, or third device the wearable device is a target device using the first proximity data, the second proximity data, and the third proximity data, wherein the target device is a device to which the wearable device is pointing during movement of the wearable device pointing; and
sending, by the processor, a message to the target device via the radio.

### Claim 16 (depends on 15)

The method of claim 15, wherein discovering the first device comprises discovering the first device using a first protocol, wherein discovering the second device comprises discovering the second device using a second protocol that is different than the first protocol.

### Claim 17 (depends on 15)

The method of claim 15, wherein discovering the first device, the second device, and the third device comprises discovering the first device, the second device, and the third device using a multi-protocol stack, the multi-protocol stack performing a scan on each channel of each wireless protocol of a plurality of wireless protocols.

### Claim 18 (depends on 15)

The method of claim 15, further comprising verifying, by the processor, that the first device, the second device, and the third device against devices registered to a user account in a device database.

### Claim 19 (depends on 15)

The method of claim 15, further comprising:
detecting, by the processor during an idle mode of the wearable device before the discovering the first device, an initial motion of the wearable device using the motion data from the motion sensor;
transitioning, by the processor, from the idle mode to a discovery mode;
initiating, by the processor during the discovery mode, a first discovery process with a first protocol to discover any devices using the first protocol; and
initiating, by the processor during the discovery mode and after the first discovery process, a second discovery process with a second protocol to discover any devices using the second protocol, wherein the first protocol is different than the second protocol.

### Claim 20 (depends on 15)

The method of claim 15, further comprising:
receiving, by a multi-protocol stack of the processor, a first packet from a multi-mode physical (PHY) layer of the radio, wherein the radio is a single radio; and
sending, by the multi-protocol stack, a second packet to the multi-mode PHY layer.

### Claim 21 (depends on 15)

The method of claim 15, wherein sending the message comprises:
retrieving, by the processor, a controlling profile associated with the first device in a device database;
retrieving, by the processor, an link key and a network key associated with the first device in a key store;
generating, by a multi-protocol stack of the processor, the message with the using the network key and the link key, wherein generating comprises encrypting the message using the network key and the link key.
