# US 2021/0174112 A1 · Deep learning-based object recognition system and method using pir sensor

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174112 A1
- Document ID: US20210174112A1
- Status: Published application
- Kind: A1
- Application number: 17/082,225
- Filed: 2020-10-28
- Published: 2021-06-10
- Earliest claimed: 2019-12-05
- Assignees: Pusan National University Industry-University Cooperation Foundation
- Applicants: Pusan National University Industry-University Cooperation Foundation
- Inventors: Jongdeok KIM; Jaemin LEE; Donghyun KIM
- Web page: https://patentgrep.com/patent/US20210174112A1

## CPC

- G08B 13/191: using pyroelectric sensor means
- G06F 18/213: Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/09: Supervised learning
- G06V 10/82: using neural networks
- G06V 20/30: in albums, collections or shared content, e.g. social network photos or video
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G08B 29/186: {Fuzzy logic; neural networks}
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Disclosed is a deep learning-based object recognition system which includes a data collection process executor configured to perform a data collection process of collecting values of the PIR sensor according to a sampling period; a data classification process executor configured to perform a data classification process of inputting the collected values of the PIR sensor to a model of an artificial neural network and transmitting a result of the inputting the collected values to a cloud using a RESTful API; and an object recognition cloud system configured to store information received from the data classification process executor in a database, transmit the information when a web application requests the information, and represent information collected and classified by devices using the RESTful API.

## Claims

### Claim 1

A deep learning-based object recognition system using a passive infra-red (PIR) sensor, the system comprising:
a data collection process executor configured to perform a data collection process of collecting values of the PIR sensor according to a sampling period;
a data classification process executor configured to perform a data classification process of inputting the collected values of the PIR sensor to a model of an artificial neural network and transmitting a result of the inputting the collected values to a cloud using a RESTful API; and
an object recognition cloud system configured to store information received from the data classification process executor in a database, transmit the information when a web application requests the information, and represent information collected and classified by devices using the RESTful API,
wherein the data process executor is further configured to limit two or more duplicate values so as not to be produced to prevent overload on a microprocessor due to calculation by the artificial neural network,
a calculation time tn of the data classification process is defined as tn=taftern−tprevn,
a limitation of a number of the data classification processes according to time is defined by taftern<tprevn+1, and
the data collection process executor, the data classification process executor, and the object recognition cloud system are each implemented via at least one processor.

### Claim 2 (depends on 1)

The system of claim 1, wherein the PIR sensor provides an output on the basis of a voltage scale within a specific area to produce frequencies by performing a fast Fourier transform (FFT) on an analog signal sample and use the frequencies as a convolutional neural network (CNN) feature vector.

### Claim 3 (depends on 1)

The system of claim 1, wherein the object recognition cloud system comprises:
a RESTful API framework configured to store information received from the data classification process executor in the database and transmit the information when the web application requests the information; and
a monitoring web application configured to represent information collected and classified by the devices using the RESTful API.

### Claim 4 (depends on 1)

The system of claim 1, wherein the data collection process executor inputs the collected values of the PIR sensor to the model of the artificial neural network, and
the artificial neural network is trained with signal patterns of motions of a human and an animal on the basis of a difference in waveform between infrared rays from a human and an animal to identify whether a recognized object is a human or an animal and to classify input data as one of three results, including no object, an animal, and a human, and indicates a distance to the object.

### Claim 5

A deep learning-based object recognition method using a passive infra-red (PIR) sensor, the method comprising:
performing a data collection process of collecting values of the PIR sensor according to a sampling period;
performing a data classification process of inputting collected values of the PIR sensor to a model of an artificial neural network and transmitting a result of the inputting the collected values to a cloud using a RESTful API; and
recognizing an object by storing information received in the data classification process in a database, transmitting the information when a web application requests the information, and representing information collected and classified by devices using the RESTful API,
wherein the performing of the data classification process comprises limiting two or more duplicate values so as not to be produced to prevent overload on a microprocessor due to calculation by the artificial neural network,
a calculation time tn of the data classification process is defined as tn=taftern−tprevn, and
a limitation of a number of the data classification processes according to time is defined by taftern<tprevn+1.

### Claim 6 (depends on 5)

The method of claim 5, wherein the performing of the data collection process comprises inputting the collected values of the PIR sensor to the model of the artificial neural network and training the artificial neural network with signal patterns of motions of a human and an animal on the basis of a difference in waveform between infrared rays from a human and an animal to identify whether the recognized object is a human or an animal and to classify input data as one of three results, including no object, an animal, and a human, and indicate a distance to the object.

### Claim 7

(canceled)

### Claim 8 (depends on 5)

The method of claim 5, wherein, when a range of tn is i±α ms, the data collection process is performed at time intervals longer than i±α ms, which is a maximum value of tn, to prevent redundant creation of the data collection process and is created at time intervals of i±α ms.

### Claim 9 (depends on 5)

The method of claim 5, wherein the artificial neural network comprises:
a data feature extracting neural network trained with waveforms of infrared rays from a human and an animal, which are receivable by the PIR sensor, to extract data features using a convolutional neural network (CNN) and cope with a change of data values due to noise and environmental changes; and
a feature correlation identifying neural network configured to identify a correlation on the basis of several features extracted from one data sample using a recurrent neural network (RNN) and to derive a final result.

### Claim 10 (depends on 5)

The method of claim 5, wherein the final result is classified as one of three results, including a human, an animal, and no object, and represents a distance to the object,
wherein the distance of the final result is continuous data and is obtained according to a linear regression,
the human, the animal, and the no object each represent a result corresponding to 0 or 1 according to a presence or absence of an object and are obtained according to logistic regression, and
an algorithm for object recognition is configured as a combination of the linear regression and the logistic regression.

### Claim 11 (depends on 10)

The method of claim 10, wherein cost functions according to the linear regression and the logistic regression are different, and a final cost function is calculated by adding values of cost functions of the linear regression and the logical regression,
wherein the cost functions of the linear regression and the logistic regression
Costlinear=(ylinear−ŷlinear)2
are expressed by:
Costlogistic=−ylogistic log(ŷlogistic)−(1−ylogistic)log(1−ŷlogistic),
and
the final cost function is calculated by:
J

(
θ
)
=
1
m

∑
i
=
1
m


[
Cost
linear
+
Cost
logistic
]
.
