# US 11,077,826 B2 · Method of processing ambient radio frequency data for activity recognition

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,077,826 B2
- Document ID: US11077826B2
- Status: Granted
- Kind: B2
- Application number: 16/739,854
- Filed: 2020-01-10
- Published: 2020-07-16
- Granted: 2021-08-03
- Earliest claimed: 2019-01-11
- Assignees: Carrier Corporation
- Applicants: Carrier Corporation
- Inventors: Junyang Lin; Zhen Jia; Jie Xi; Tianyuan Chen; Yuanjing Sun
- Web page: https://patentgrep.com/patent/US11077826B2

## CPC

- G08B 13/187: by interference of a radiation field
- B60R 25/1004: {Alarm systems characterised by the type of sensor, e.g. current sensing means (using door lock sensors B60R25/1001)}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/084: Backpropagation, e.g. using gradient descent
- G06N 20/00: Machine learning
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G08B 13/2491: {Intrusion detection systems, i.e. where the body of an intruder causes the interference with the electromagnetic field}
- H04B 7/0626: {Channel coefficients, e.g. channel state information [CSI]}
- H04W 24/08: Testing, {supervising or monitoring} using real traffic
- G06F 2218/02: Preprocessing
- G06F 2218/04: Denoising
- G06F 2218/12: Classification; Matching
- G06T 2207/10: Image acquisition modality
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- B60R: VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04B: TRANSMISSION
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method of operating an activity recognition system includes capturing ambient radio frequency (RF) data by an RF sniffer. The ambient RF data is then received by a processor. The processor reduces noise content of the ambient RF data. Background is then subtracted from the ambient RF data by the processor. The processed ambient RF data is then converted into an image by the processor. The system generates successive images for each one of a plurality of time intervals. An image processing algorithm, stored in a storage medium and executed by the processor, is applied to the plurality of successive images to determine activity recognition.

## Claims

### Claim 1

A method of operating an activity recognition system comprising:
capturing ambient radio frequency (RF) data by an RF sniffer;
receiving the ambient RF data by a processor;
reducing noise content of the ambient RF data by the processor;
subtracting background from the ambient RF data by the processor;
converting the ambient RF data with reduced noise and subtracted background into an image by the processor;
generating a successive image for each one of a plurality of time intervals by the processor; and
apply an image processing algorithm storing in a storage medium and executed by the processor to each successive image to determine activity recognition, wherein the noise content of the ambient RF data is reduced by removing a mean value of multiple Channel State information (CSI) subcarriers at the same time index to subtract common mode noise.

### Claim 2

A method of operating an activity recognition system comprising:
capturing ambient radio frequency (RF) data by an RF sniffer;
receiving the ambient RF data by a processor;
reducing noise content of the ambient RF data by the processor;
subtracting background from the ambient RF data by the processor;
converting the ambient RF data with reduced noise and subtracted background into an image by the processor;
generating a successive image for each one of a plurality of time intervals by the processor; and
apply an image processing algorithm storing in a storage medium and executed by the processor to each successive image to determine activity recognition, wherein the subtracting background includes converting the ambient RF data to a first order derivative of time.

### Claim 3 (depends on 1)

The method set forth in claim 1, wherein the subtracting background includes converting the ambient RF data to a first order derivative of time.

### Claim 4 (depends on 3)

The method set forth in claim 3, wherein the conversion to an image includes the combination of RF data from a plurality of antenna channels.

### Claim 5

A building system comprising:
a wireless radio device including a transmitting component configured to transmit a radio frequency (RF), and a receiving component configured to receive the RF to accomplish a primary task; and
an activity recognition system configured to perform an activity recognition task, the activity recognition system including a sniffer configured to sample and measure ambient RF signals over time, control circuitry including one or more processors and one or more storage mediums, RF background data stored in at least one of the one or more storage mediums and indicative of no activity, a computer instruction stored in at least one of the one or more storage mediums and executed by at least one of the one or more processors, wherein the computer instruction is configured to process the measured ambient RF signals, subtract background from ambient RF data indicative of the ambient RF signals, convert the process ambient RF signals to a plurality of successive images, and apply an image-based algorithm to compare the plurality of successive images to the RF background data, and thereby determine activity recognition, wherein the noise content of the ambient RF data is reduced by removing a mean value of multiple Channel State information (CSI) subcarriers at the same time index to subtract common mode noise.

### Claim 6 (depends on 5)

The building system set forth in claim 5, wherein the transmitting component, the receiving component, and the sniffer are located in a building.

### Claim 7 (depends on 6)

The building system set forth in claim 6, wherein the sniffer is one of a plurality of sniffers each located in a respective region of a plurality of regions of the building.

### Claim 8 (depends on 6)

The building system set forth in claim 6, wherein the wireless radio device is one of a plurality of wireless radio devices each transmitting respective RF signals sampled by the sniffer.

### Claim 9 (depends on 5)

The building system set forth in claim 5, wherein the wireless radio device is a WiFi device.

### Claim 10 (depends on 5)

The building system set forth in claim 5, wherein the activity recognition system is an intruder alert system.