# US 2021/0231775 A1 · System and method for smart device control using radar

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0231775 A1
- Document ID: US20210231775A1
- Status: Published application
- Kind: A1
- Application number: 16/773,116
- Filed: 2020-01-27
- Published: 2021-07-29
- Earliest claimed: 2020-01-27
- Assignees: Plato Systems, Inc.
- Applicants: Plato Systems, Inc.
- Inventors: Aria Pezeshk; Mashhour Solh; Mohammad Amin Arbabian
- Web page: https://patentgrep.com/patent/US20210231775A1

## CPC

- G01S 7/415: {Identification of targets based on measurements of movement associated with the target}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G01S 7/285: Receivers
- G01S 7/352: {Receivers}
- G01S 7/417: {involving the use of neural networks}
- G01S 13/867: {Combination of radar systems with cameras}
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/172: {Classification, e.g. identification}
- H04L 12/2823: {Reporting information sensed by appliance or service execution status of appliance services in a home automation network (device-related reporting H04L43/065; arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom H04Q9/00)}
- G01S 7/356: {involving particularities of FFT processing}
- H04L 2012/2849: {Audio/video appliances}
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)

## Abstract

Systems and methods for smart device control using radar are disclosed. According to some aspects, a machine receives, using a millimeter-wave multiple antenna array, a radar signal. The machine preprocesses the radar signal to generate radar metadata. The machine determines, using a trained machine learning engine and based on at least the radar metadata, a moving entity and a movement type. The machine identifies, based on at least the determined moving entity and the determined movement type, a smart device and an action for the smart device to take in response to the movement type by the moving entity. The machine transmits, to the smart device, a control signal for the identified action.

## Claims

### Claim 1

A system comprising:
processing circuitry; and
a memory storing instructions which, when executed by the processing circuitry, cause the processing circuitry to perform operations comprising:
receiving, using a millimeter-wave multiple antenna array, a radar signal;
preprocessing the radar signal to generate radar metadata;
determining, using a trained machine learning engine and based on at least the radar metadata, a moving entity and a movement type;
identifying, based on at least the determined moving entity and the determined movement type, a smart device and an action for the smart device to take in response to the movement type by the moving entity; and
communicating, to the smart device, a control signal for the identified action.

### Claim 2 (depends on 1)

The system of claim 1, wherein the moving entity comprises one or more of: a specific person, a non-specific person, an animal, a moving object, a group of moving persons, animals or objects.

### Claim 3 (depends on 1)

The system of claim 1, the operations further comprising:
receiving, using an imaging unit and in conjunction with the radar signal, a camera signal;
preprocessing the camera signal to generate camera metadata, wherein the moving entity and the movement type are determined based on the camera metadata.

### Claim 4 (depends on 3)

The system of claim 3, wherein the imaging unit comprises two or more cameras, and wherein the camera metadata comprises depth data.

### Claim 5 (depends on 1)

The system of claim 1, the operations further comprising:
receiving, using a microphone and in conjunction with the radar signal, an audio signal;
preprocessing the audio signal to generate audio metadata, wherein the moving entity is determined based on the audio metadata.

### Claim 6 (depends on 1)

The system of claim 1, wherein the smart device comprises one or more of: a microphone, a camera, a lamp, a door, a lock, an audio player, a television, and an alarm.

### Claim 7 (depends on 1)

The system of claim 1, wherein:
the radar signal comprises one or more chirps, pulses or orthogonal frequency-division multiplexing (OFDM), frequency modulated continuous wave (FMCW) or step-frequency continuous wave (SFCW) signals; and
preprocessing the radar signal comprises computing a range, a velocity, or an angle of the moving entity using a fast Fourier transform (FFT).

### Claim 8 (depends on 1)

The system of claim 1, the operations further comprising:
storing, in the memory, a map of a space surrounding the millimeter-wave multiple antenna array, wherein the smart device is identified based on a stored position of the smart device on the map, the determined moving entity, and the determined movement type.

### Claim 9 (depends on 1)

The system of claim 1, wherein determining the moving entity and the movement type is based on Micro-Doppler or Range Doppler Angle or point cloud data extraction.

### Claim 10 (depends on 1)

The system of claim 1, wherein the trained machine learning engine comprises at least one convolutional neural network (CNN) and at least one recurrent neural network (RNN).

### Claim 11 (depends on 1)

The system of claim 1, wherein the trained machine learning engine comprises a convolutional neural network (CNN), the CNN comprising a plurality of convolution layers and a plurality of pooling layers.

### Claim 12 (depends on 1)

The system of claim 1, further comprising:
the millimeter-wave multiple antenna array; and
the smart device.

### Claim 13

A non-transitory machine-readable medium storing instructions which, when executed by a computing machine, cause the computing machine to perform operations comprising:
receiving, using a millimeter-wave multiple antenna array, a radar signal;
preprocessing the radar signal to generate radar metadata;
determining, using a trained machine learning engine and based on at least the radar metadata, a moving entity and a movement type;
identifying, based on at least the determined moving entity and the determined movement type, a smart device and an action for the smart device to take in response to the movement type by the moving entity; and
communicating, to the smart device, a control signal for the identified action.

### Claim 14 (depends on 13)

The machine-readable medium of claim 13, wherein the moving entity comprises one or more of: a specific person, a non-specific person, an animal, a moving object, a group of moving persons, animals or objects.

### Claim 15 (depends on 13)

The machine-readable medium of claim 13, the operations further comprising:
receiving, using an imaging unit and in conjunction with the radar signal, a camera signal;
preprocessing the camera signal to generate camera metadata, wherein the moving entity and the movement type are determined based on the camera metadata.

### Claim 16 (depends on 15)

The machine-readable medium of claim 15, wherein the imaging unit comprises two or more cameras, and wherein the camera metadata comprises depth data.

### Claim 17 (depends on 13)

The machine-readable medium of claim 13, the operations further comprising:
receiving, using a microphone and in conjunction with the radar signal, an audio signal;
preprocessing the audio signal to generate audio metadata, wherein the moving entity is determined based on the audio metadata.

### Claim 18 (depends on 13)

The machine-readable medium of claim 13, wherein the smart device comprises one or more of: a microphone, a camera, a lamp, a door, a lock, an audio player, a television, and an alarm.

### Claim 19 (depends on 13)

The machine-readable medium of claim 13, wherein:
the radar signal comprises one or more chirps, pulses or orthogonal frequency-division multiplexing (OFDM) signals; and
preprocessing the radar signal comprises computing a range, a velocity, or an angle of the moving entity using a fast Fourier transform (FFT).

### Claim 20

A method comprising:
receiving, using a millimeter-wave multiple antenna array, a radar signal;
preprocessing the radar signal to generate radar metadata;
determining, using a trained machine learning engine and based on at least the radar metadata, a moving entity and a movement type;
identifying, based on at least the determined moving entity and the determined movement type, a smart device and an action for the smart device to take in response to the movement type by the moving entity; and
communicating, to the smart device, a control signal for the identified action.
