# US 12,769,299 B2 · Methods and systems for detecting operating conditions of an industrial machine using the industrial internet of things > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,769,299 B2 - Document ID: US12769299B2 - Status: Granted - Kind: B2 - Application number: 16/685,372 - Filed: 2019-11-15 - Published: 2020-04-30 - Granted: 2026-10-20 - Earliest claimed: 2018-05-07 - Assignees: Strong Force IOT Portfolio 2016, LLC - Applicants: Strong Force IOT Portfolio 2016, LLC - Inventors: Charles Howard Cella; Gerald William Duffy, Jr.; Jeffrey P. McGuckin; Mehul Desai - Web page: https://patentgrep.com/patent/US12769299B2 ## CPC - G05B 23/0283: {Predictive maintenance, e.g. involving the monitoring of a system and, based on the monitoring results, taking decisions on the maintenance schedule of the monitored system; Estimating remaining useful life [RUL] (preventive maintenance, i.e. planning maintenance according to the available resources without monitoring the system G06Q10/06)} - G01M 13/028: Acoustic or vibration analysis - G01M 13/045: Acoustic or vibration analysis - G05B 13/028: {using expert systems only} - G05B 19/4183: {characterised by data acquisition, e.g. workpiece identification} - G05B 19/4184: {characterised by fault tolerance, reliability of production system} - G05B 19/41845: {characterised by system universality, reconfigurability, modularity} - G05B 19/4185: {characterised by the network communication} - G05B 19/41865: {characterised by job scheduling, process planning, material flow} - G05B 19/41875: {characterised by quality surveillance of production} - G05B 23/0221: {Preprocessing measurements, e.g. data collection rate adjustment; Standardization of measurements; Time series or signal analysis, e.g. frequency analysis or wavelets; Trustworthiness of measurements; Indexes therefor; Measurements using easily measured parameters to estimate parameters difficult to measure; Virtual sensor creation; De-noising; Sensor fusion; Unconventional preprocessing inherently present in specific fault detection methods like PCA-based methods} - G05B 23/0229: {knowledge based, e.g. expert systems; genetic algorithms} - G05B 23/024: {Quantitative history assessment, e.g. mathematical relationships between available data; Functions therefor; Principal component analysis [PCA]; Partial least square [PLS]; Statistical classifiers, e.g. Bayesian networks, linear regression or correlation analysis; Neural networks} - G05B 23/0264: {Control of logging system, e.g. decision on which data to store; time-stamping measurements} - G05B 23/0286: {Modifications to the monitored process, e.g. stopping operation or adapting control} - G05B 23/0289: {Reconfiguration to prevent failure, e.g. usually as a reaction to incipient failure detection} - G05B 23/0291: {Switching into safety or degraded mode, e.g. protection and supervision after failure} - G05B 23/0294: {Optimizing process, e.g. process efficiency, product quality} - G05B 23/0297: {Reconfiguration of monitoring system, e.g. use of virtual sensors; change monitoring method as a response to monitoring results} - G06F 16/24: Querying - G06F 18/2178: {based on feedback of a supervisor} - G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] - G06N 3/02: Neural networks - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/047: Probabilistic or stochastic networks - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/088: Non-supervised learning, e.g. competitive learning - G06N 5/046: Forward inferencing; Production systems - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - G06T 7/0004: {Industrial image inspection} - H04B 17/309: Measuring or estimating channel quality parameters - H04B 17/318: Received signal strength - H04L 1/0002: {by adapting the transmission rate} - H04L 1/0041: {Arrangements at the transmitter end} - H04L 1/18: Automatic repetition systems, e.g. Van Duuren systems - H04L 67/1097: for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS] - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - G05B 2219/32287: Medical, chemical, biological laboratory - G05B 2219/35001: Data input, data handling, programming, monitoring of nc - G05B 2219/37337: Noise, acoustic emission, sound - G05B 2219/37351: Detect vibration, ultrasound - G05B 2219/37434: Measuring vibration of machine or workpiece or tool - G05B 2219/40115: Translate goal to task program, use of expert system - G05B 2219/45004: Mining - G05B 2219/45129: Boring, drilling - G06F 16/2379: {Updates performed during online database operations; commit processing} - G06F 16/245: Query processing - G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming - H04L 5/0064: {Rate requirement of the data, e.g. scalable bandwidth, data priority} - Y02P 80/10: Efficient use of energy, e.g. using compressed air or pressurized fluid as energy carrier - Y02P 90/02: Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS] - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G01M: TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04B: TRANSMISSION - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - Y02P: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS ## Abstract Methods and systems for detecting operating characteristics of an industrial machine in which the systems include at least one data capture device configured to capture raw data of a point of interest of the industrial machine and a computer vision system. The computer vision system can generate one or more image data sets using the raw data captured, identify one or more values corresponding to a portion of the industrial machine within the point of interest represented by the one or more image data sets, compare the one or more values to corresponding predicted values, generate a variance data set based on the comparison of the one or more values and the corresponding predicted values, detect an operating characteristic of the industrial machine based on the variance data, and generate data indicating the detection of the operating characteristic. ## Claims ### Claim 1 A system for detecting operating characteristics of an industrial machine, comprising: at least one data capture device configured to capture raw data of a point of interest of the industrial machine; a computer vision system that: generates one or more image data sets using the raw data captured, identifies one or more values corresponding to a portion of the industrial machine within the point of interest represented by the one or more image data sets, compares the one or more values to corresponding predicted values, wherein: (i) the predicted values correspond to characteristics of the portion of the industrial machine that indicate that the industrial machine is operating within acceptable or expected tolerances, and (ii) the predicted values are generated based on training data sets generated by a deep learning system that correlates data with an indicator of an outcome, generates a variance data set based on the comparison of the one or more values and the corresponding predicted values, the variance data set representing a difference between the one or more values and the corresponding predicted values, detects an operating characteristic of the industrial machine based on the variance data by utilizing a classification model trained to identify the operating characteristic based on variance data, wherein the operating characteristic represents a possible or present issue relating to an operation of the industrial machine, and generates data indicating the detection of the operating characteristic; and a predictive maintenance platform that processes the data indicating the detection of the operating characteristic to identify a maintenance action representing an action which may be taken to prevent or resolve the possible or present issue relating to an operation of the industrial machine. ### Claim 2 (depends on 1) The system of claim 1, wherein the computer vision system generates a signal indicative of the data indicating the detection of the operating characteristic, wherein the predictive maintenance platform predicts the possible or present issue based on the signal. ### Claim 3 (depends on 1) The system of claim 1, further comprising: a mobile data collector configured to perform the maintenance action, wherein the predictive maintenance platform or the computer vision system transmits a signal indicative of the maintenance action to the mobile data collector to cause the mobile data collector to perform the maintenance action. ### Claim 4 (depends on 3) The system of claim 3, wherein the at least one data capture device captures the raw data in response to the mobile data collector recording a state-related measurement of the industrial machine. ### Claim 5 (depends on 4) The system of claim 4, wherein the state-related measurement of the industrial machine relates to a vibration of at least the portion of the industrial machine captured using at least one vibration sensor of the mobile data collector. ### Claim 6 (depends on 5) The system of claim 5, wherein the mobile data collector transmits a signal to at least one of the at least one data capture device or the computer vision system to cause the at least one data capture device to capture the raw data, wherein the signal is generated by: receiving, at a computing device, vibration data representative of the vibration of at least the portion of the industrial machine from the mobile data collector; determining, by the computing device, a frequency of the captured vibration by processing the captured vibration data; determining, by the computing device and based on the frequency, a segment of a multi-segment vibration frequency spectra that bounds the captured vibration; calculating, by the computing device, a severity unit for the captured vibration based on the determined segment; and causing the mobile data collector to generate the signal based on the severity unit. ### Claim 7 (depends on 6) The system of claim 6, wherein calculating the severity unit for the captured vibration based on the determined segment comprises: mapping the captured vibration to the severity unit based on the determined segment by: mapping the captured vibration to a first severity unit when the frequency of the captured vibration corresponds to a below a low-end knee threshold-range of the multi-segment vibration frequency spectra; mapping the captured vibration to a second severity unit when the frequency of the captured vibration corresponds to a mid-range of the multi-segment vibration frequency spectra; and mapping the captured vibration to a third severity unit when the frequency of the captured vibration corresponds to an above a high-end knee threshold-range of the multi-segment vibration frequency spectra. ### Claim 8 (depends on 5) The system of claim 5, wherein the predictive maintenance platform uses a distributed ledger to track maintenance transactions related to the industrial machine, the distributed ledger storing transaction records corresponding to the maintenance transactions. ### Claim 9 (depends on 8) The system of claim 8, wherein the predictive maintenance platform generates a new transaction record in response to the transmission of the signal by the mobile data collector. ### Claim 10 (depends on 8) The system of claim 8, wherein the predictive maintenance platform generates a new transaction record in response to the generation of the data indicating the detection of the operating characteristic by the computer vision system. ### Claim 11 (depends on 5) The system of claim 5, wherein the at least one vibration sensor of the mobile data collector captures the vibration based on a waveform derived from a vibration envelope associated with at least the portion of the industrial machine. ### Claim 12 (depends on 3) The system of claim 3, wherein the mobile data collector is a mobile robot. ### Claim 13 (depends on 1) The system of claim 1, wherein the predictive maintenance platform uses a distributed ledger to track maintenance transactions related to the industrial machine, the distributed ledger storing transaction records corresponding to the maintenance transactions. ### Claim 14 (depends on 13) The system of claim 13, wherein the predictive maintenance platform generates a new transaction record in response to the capturing of the raw data by the at least one data capture device. ### Claim 15 (depends on 13) The system of claim 13, wherein the predictive maintenance platform generates a new transaction record in response to the generation of the data indicating the detection of the operating characteristic by the computer vision system. ### Claim 16 (depends on 1) The system of claim 1, wherein the predictive maintenance platform trains a machine learning aspect to detect possible or present issues similar to the possible or present issue relating to the operation of the industrial machine based on at least one of the operating characteristic or the maintenance action. ### Claim 17 (depends on 1) The system of claim 1, further comprising: a visual analyzer including an intelligent system that analyzes the data indicating the detection of the operating characteristic to train a machine learning aspect associated with the computer vision system by: training the machine learning aspect using a training data set comprising one or more of the operating characteristic, the data indicating the detection of the operating characteristic, the raw data of the point of interest of the industrial machine, or the one or more image data sets. ### Claim 18 (depends on 17) The system of claim 17, further comprising: a training data database that stores the training data set, wherein the visual analyzer trains the machine learning aspect associated with the computer vision system by retrieving the training data set from the training data database. ### Claim 19 (depends on 1) The system of claim 1, wherein the operating characteristic includes a vibration of a component of the industrial machine. ### Claim 20 (depends on 1) The system of claim 1, wherein the operating characteristic includes a shape of a component of the industrial machine or wherein the operating characteristic includes a size of a component of the industrial machine. ### Claim 21 (depends on 1) The system of claim 1, wherein the operating characteristic includes a deflection of a component of the industrial machine. ### Claim 22 (depends on 1) The system of claim 1, wherein the operating characteristic includes an electromagnetic emission of a component of the industrial machine. ### Claim 23 (depends on 1) The system of claim 1, wherein the operating characteristic includes at least one of: (i) a temperature of a component of the industrial machine, (ii) a temperature of a gas within a component of the industrial machine, (iii) a temperature of a liquid within a component of the industrial machine, or (iv) a temperature of a solid within a component of the industrial machine. ### Claim 24 (depends on 1) The system of claim 1, wherein the operating characteristic includes at least one of: (i) a pressure within a component of the industrial machine, (ii) a pressure of a gas within a component of the industrial machine, or (iii) a pressure of a liquid within a component of the industrial machine. ### Claim 25 (depends on 1) The system of claim 1, wherein the operating characteristic includes at least one of: (i) a density of a gas within a component of the industrial machine, (ii) a density of a liquid within a component of the industrial machine, or (iii) a density of a solid within a component of the industrial machine. ### Claim 26 (depends on 1) The system of claim 1, wherein the operating characteristic includes a density of a component manufactured by the industrial machine. ### Claim 27 (depends on 26) The system of claim 26, wherein the component includes a part for a vehicle. ### Claim 28 (depends on 26) The system of claim 26, wherein the component includes a part for a bike. ### Claim 29 (depends on 26) The system of claim 26, wherein the component includes a bike chain. ### Claim 30 (depends on 26) The system of claim 26, wherein the component includes a gasket. ### Claim 31 (depends on 26) The system of claim 26, wherein the component includes at least one of: (i) a fastener, (ii) a part for a screw, or (iii) a part for a bolt. ### Claim 32 (depends on 26) The system of claim 26, wherein the component includes a part for a printed circuit board. ### Claim 33 (depends on 26) The system of claim 26, wherein the component includes at least one of: (i) a part for a capacitor, (ii) a part for a resistor, or (iii) a part for an inductor. ### Claim 34 (depends on 1) The system of claim 1, wherein the operating characteristic includes at least one of: (i) a chemical structure of a gas within a component of the industrial machine, (ii) a chemical structure of a liquid within a component of the industrial machine, or (iii) a chemical structure of a solid within a component of the industrial machine. ### Claim 35 (depends on 1) The system of claim 1, wherein the operating characteristic includes a chemical structure of a component manufactured by the industrial machine. ### Claim 36 (depends on 35) The system of claim 35, wherein the component includes a part for a vehicle. ### Claim 37 (depends on 35) The system of claim 35, wherein the component includes a part for a bike. ### Claim 38 (depends on 35) The system of claim 35, wherein the component includes a bike chain. ### Claim 39 (depends on 35) The system of claim 35, wherein the component includes a gasket. ### Claim 40 (depends on 35) The system of claim 35, wherein the component includes at least one of: (i) a fastener, (ii) a part for a screw, or (iii) a part for a bolt. ### Claim 41 (depends on 35) The system of claim 35, wherein the component includes a part for a printed circuit board. ### Claim 42 (depends on 35) The system of claim 35, wherein the component includes at least one of: (i) a part for a capacitor, (ii) a part for a resistor, or (iii) a part for an inductor. ### Claim 43 (depends on 1) The system of claim 1, wherein the data capture device includes an image capture device. ### Claim 44 (depends on 1) The system of claim 1, wherein the data capture device includes a camera. ### Claim 45 (depends on 1) The system of claim 1, wherein the data capture device includes a data measurement device. ### Claim 46 (depends on 1) The system of claim 1, wherein the data capture device includes a sensor. ### Claim 47 (depends on 1) The system of claim 1, wherein the data capture device includes a full spectrum camera. ### Claim 48 (depends on 1) The system of claim 1, wherein the data capture device includes a radiation imaging device. ### Claim 49 (depends on 1) The system of claim 1, wherein the data capture device includes an X-ray imaging device. ### Claim 50 (depends on 1) The system of claim 1, wherein the data capture device includes a non-visible light data capture device. ### Claim 51 (depends on 1) The system of claim 1, wherein the data capture device includes a visible light data capture device. ### Claim 52 (depends on 1) The system of claim 1, wherein the data capture device includes a sonic data capture device. ### Claim 53 (depends on 1) The system of claim 1, wherein the data capture device includes light imaging, detection, and ranging device. ### Claim 54 (depends on 1) The system of claim 1, wherein the data capture device includes a point cloud data capture device. ### Claim 55 (depends on 1) The system of claim 1, wherein the data capture device includes an infrared inspection device. ### Claim 56 (depends on 18) The system of claim 18, wherein the data capture device includes an image capture device. ### Claim 57 (depends on 1) The system of claim 1, wherein the data capture device includes at least one of: (i) a pressure sensor, (ii) a temperature sensor, or (iii) a chemical sensor. ### Claim 58 (depends on 1) The system of claim 1, wherein the data capture device includes a stand-alone device. ### Claim 59 (depends on 1) The system of claim 1, wherein the data capture device includes at least one of: (i) a mobile device, (ii) a smart phone, or (iii) a tablet. ### Claim 60 (depends on 1) The system of claim 1, wherein the raw data includes raw image data. ### Claim 61 (depends on 1) The system of claim 1, wherein the raw data includes raw measurement data. ### Claim 62 (depends on 1) The system of claim 1, wherein the portion of the industrial machine within the point of interest includes a component of the industrial machine. ### Claim 63 (depends on 1) The system of claim 1, wherein the portion of the industrial machine within the point of interest includes a belt of the industrial machine. ### Claim 64 (depends on 1) The system of claim 1, wherein the portion of the industrial machine within the point of interest includes a component manufactured by the industrial machine. ### Claim 65 (depends on 1) The system of claim 1, wherein the portion of the industrial machine within the point of interest includes a bike chain manufactured by the industrial machine.