# US 11,299,371 B2 · System and method for detecting elevator maintenance behavior in elevator hoistway > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,299,371 B2 - Document ID: US11299371B2 - Status: Granted - Kind: B2 - Application number: 16/668,725 - Filed: 2019-10-30 - Published: 2020-04-30 - Granted: 2022-04-12 - Earliest claimed: 2018-10-30 - Assignees: Otis Elevator Company - Applicants: Otis Elevator Company - Inventors: Yuanjing Sun; Jie Xi; JunYang Lin; TianYuan Chen; Zhen Jia - Web page: https://patentgrep.com/patent/US11299371B2 ## CPC - B66B 5/0062: {by devices, being operable or not, mounted on the elevator car} - B66B 5/0087: {Devices facilitating maintenance, repair or inspection tasks (devices incorporated in the buffer B66B5/288; railings on top of the car B66B11/0226)} - B66B: ELEVATORS; ESCALATORS OR MOVING WALKWAYS ({apparatus for raising or lowering persons on theatrical stages or the like A63J5/12} ; funicular railbound systems with rigid ground-supported tracks and cable traction, e.g. cliff railways, B61B9/00; arrangements of ammunition handlers in vessels B63G3/00; hoists, lifts, or conveyors for loading or unloading in general B65G; braking or detent devices controlling normal movements of winding drums or barrels B66D; ship-lifting devices E02C; garages for many vehicles with mechanical means for lifting vehicles E04H6/12; hoists for feeding ammunition or projectiles to launching apparatus or to loading mechanisms F41A9/00) ## Abstract A system and method for detecting elevator maintenance behaviors in an elevator hoistway. The system of the invention comprises: an emitting component for emitting wireless signals including channel state information (CSI) into the elevator hoistway; a receiving component for receiving the CSI from the elevator hoistway; and a maintenance behavior detecting device, which comprises: a memory that stores CIS image recognition models constructed corresponding to the predefined elevator maintenance behaviors and a corresponding computer program executable on a processor, and the processor configured to be able to execute the computer program to implement the following operations: processing data of the received CSI to obtain a CSI image corresponding to the elevator maintenance behavior detected, and inputting the CSI image to the CSI image recognition model for analysis and processing so as to detect the elevator maintenance behavior. ## Claims ### Claim 1 A system for detecting elevator maintenance behaviors of a maintainer in an elevator hoistway, which comprises: an emitting component for emitting wireless signals including channel state information (CSI) into the elevator hoistway; a receiving component for receiving the CSI from the elevator hoistway; and a maintenance behavior detecting device coupled to the receiving component, which comprises: a memory that stores CSI image recognition models constructed corresponding to one or more predefined elevator maintenance behaviors and a corresponding computer program executable on a processor; and the processor, which is configured to be able to execute the computer program to implement the following operations: processing data of the received CSI to obtain a CSI image corresponding to the elevator maintenance behavior detected; and inputting the CSI image to the CSI image recognition model for analysis and processing so as to detect the elevator maintenance behavior of the maintainer in the elevator hoistway. ### Claim 2 (depends on 1) The system according to claim 1, wherein the operation of processing data of the received CSI to obtain CSI images corresponding to the elevator maintenance behaviors detected includes the following procedures: denoising the received CSI; removing background information from the CSI; and converting the CSI, removed the background information therefrom, corresponding to multiple subcarriers to generate a corresponding CSI image. ### Claim 3 (depends on 1) The system according to claim 1, wherein the emitting component and the receiving component are mounted on the outer top of the elevator cab in the elevator hoistway, and the emitting component and the receiving component travel synchronously with the elevator cab. ### Claim 4 (depends on 1) The system according to claim 1, wherein the emitting component is a WiFi wireless access point device, and the receiving component is a WiFi wireless network card. ### Claim 5 (depends on 1) The system according to claim 1, wherein the predefined elevator maintenance behavior includes a dangerous maintenance behavior; and the operation of inputting the CSI image to the CSI image recognition model for analysis and processing so as to detect the elevator maintenance behaviors includes: identifying whether the elevator maintenance behavior is the dangerous maintenance behavior. ### Claim 6 (depends on 1) The system according to claim 1, wherein the predefined elevator maintenance behaviors includes multiple types of maintenance behaviors; the operation of inputting the CSI image to the CSI image recognition model for analysis and processing so as to detect the elevator maintenance behavior includes: classifying the elevator maintenance behavior into a certain type of maintenance behavior. ### Claim 7 (depends on 5) The system according to claim 5, which further comprises: a reminder component which is used to send a reminder signal indicating that the elevator maintenance behavior currently being detected is the dangerous maintenance behavior. ### Claim 8 (depends on 1) The system according to claim 1, wherein the CSI image recognition model includes a behavior feature library corresponding to behavior feature models of the predefined elevator maintenance behaviors. ### Claim 9 (depends on 8) The system according to claim 8, wherein the processor is further configured to be able to execute the computer program to implement the following operations: collecting the CSI corresponding to the predefined elevator maintenance behaviors and using it as training data; and training on the basis of the training data to obtain the behavior feature models. ### Claim 10 (depends on 9) The system according to claim 9, wherein the processor is further configured to be able to execute the computer program to implement the following operations: inputting the recognized CSI image as training data into an original CSI image recognition model that has stored the behavior feature model; and updating the original CSI image recognition model using a machine learning algorithm to obtain the CSI image recognition model. ### Claim 11 (depends on 1) The system according to claim 1, wherein the processor is further configured to be able to execute the computer program to implement the following operations: corresponding the time, amplitudes of multiple subcarriers in the time domain and frequencies of multiple subcarriers in the time domain of the CSI to R, G and B components of an image and realizing a matrix representation to generate the CSI image; segmenting an image flow using a time window to obtain a series of CSI images; inputting the series of CSI images into the CSI image recognition model for analysis and processing to identify the elevator maintenance behavior. ### Claim 12 (depends on 1) The system according to claim 1, wherein the CSI image recognition model is a convolution neural network model. ### Claim 13 A method for detecting elevator maintenance behaviors of a maintainer in an elevator hoistway, which comprises: (S1) receiving channel state information (CSI), wherein the CSI is included in wireless signals emitted into the elevator hoistway; (S2) processing data of the received CSI to obtain a CSI image corresponding to the elevator maintenance behavior detected; and (S3) inputting the CSI image to a CSI image recognition model for analysis and processing so as to detect the elevator maintenance behavior of the maintainer in the elevator hoistway, wherein the CSI image recognition model is constructed corresponding to one or more predefined elevator maintenance behaviors of the maintainer in the elevator hoistway. ### Claim 14 (depends on 13) The method according to claim 13, wherein (S2) includes: a data pre-processing operation which includes denoising the received CSI; a background removing operation which removes background information from the CSI; an image converting operation which converts the CSI, removed the background information therefrom, corresponding to multiple subcarriers to generate corresponding CSI images. ### Claim 15 (depends on 13) The method according to claim 13, wherein the predefined elevator maintenance behaviors include a dangerous maintenance behavior; wherein (S3) includes identifying whether the elevator maintenance behavior is the dangerous maintenance behavior. ### Claim 16 (depends on 13) The method according to claim 13, wherein the predefined elevator maintenance behaviors include various types of maintenance behaviors; wherein (S3) includes classifying the elevator maintenance behavior into a certain type of maintenance behavior. ### Claim 17 (depends on 15) The method according to claim 15, further comprising: sending a reminder signal upon identifying the dangerous maintenance behavior. ### Claim 18 (depends on 13) The method according to claim 13, wherein the CSI image recognition model includes a behavior feature library corresponding to behavior feature models of the predefined elevator maintenance behaviors; the method further comprises: collecting the CSI corresponding to the predefined elevator maintenance behaviors and using it as training data; and training on the basis of the training data to obtain the behavior feature models. ### Claim 19 (depends on 18) The method according to claim 18, wherein the CSI image recognition model is constructed or updated through the following operations: inputting the recognized CSI image as training data into an original CSI image recognition model that has stored the behavior feature model; and updating the original CSI image recognition model using a machine learning algorithm to obtain the CSI image recognition model. ### Claim 20 (depends on 13) The method according to claim 13, wherein (S2) includes: corresponding the time, amplitudes of multiple subcarriers in the time domain and frequencies of multiple subcarriers in the time domain of the CSI to R, G and B components of an image and realizing a matrix representation to generate the CSI image; segmenting an image flow using a time window to obtain a series of CSI images; wherein (S3) includes: inputting the series of CSI images into the CSI image recognition model for analysis and processing to identify the elevator maintenance behavior. ### Claim 21 (depends on 13) The method according to claim 13, wherein the CSI image recognition model is a convolution neural network model. ### Claim 22 (depends on 13) A computer device, which comprises a memory, a processor, a CSI image recognition model stored on the memory and constructed corresponding to one or more predefined elevator maintenance behaviors, and corresponding computer programs executable on the processor, wherein the processor can execute the programs to implement the method of claim 13. ### Claim 23 (depends on 13) A computer-readable storage medium, on which a CSI image recognition model constructed corresponding to one or more predefined elevator maintenance behaviors and corresponding computer programs executable on the processor are stored, wherein said programs can be executed by the processor to implement the method of claim 13.