# US 2020/0401125 A1 · Multi-sensor suite and interactive software platform for real-time visualization and analysis of distributed environment conditions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401125 A1 - Document ID: US20200401125A1 - Status: Published application - Kind: A1 - Application number: 16/906,969 - Filed: 2020-06-19 - Published: 2020-12-24 - Earliest claimed: 2019-06-20 - Inventors: Karl F. HIRSCH - Web page: https://patentgrep.com/patent/US20200401125A1 ## CPC - G05B 23/0283: {Predictive maintenance, e.g. involving the monitoring of a system and, based on the monitoring results, taking decisions on the maintenance schedule of the monitored system; Estimating remaining useful life [RUL] (preventive maintenance, i.e. planning maintenance according to the available resources without monitoring the system G06Q10/06)} - G05B 23/0272: {Presentation of monitored results, e.g. selection of status reports to be displayed; Filtering information to the user} - G05B 2219/50393: Floor conveyor, AGV automatic guided vehicle - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract An industrial IoT system is provided that has adaptable, re-configurable, connected and integrated suite of smart, synchronized sensors and video/thermal cameras for continuous monitoring, access and control from anywhere in an industrial environment. A path sensor comprises a multi-sensor device with lookaround housing feature for monitoring conditions of conveyor brushes. A package sensor comprises a multi-sensor device with sensors adapted for mounting in different form factor carriers for monitoring conditions during package transport. A software platform integrates multiple layers of information from the sensors and video/thermal cameras and corresponding RFID locations into a single meaningful experience/assessment tool that can further be superimposed on a scaled representation of a monitored industrial environment generated from a map and dimensions for same. ## Claims ### Claim 1 A path sensor for detecting conditions of a designated moving portion of a conveyor, the conveyor portion being energized via brushes that contact a power rail, the path sensor comprising: a controller; and a sensor connected to the controller and configured to detect a condition of the brushes. ### Claim 2 (depends on 1) The path sensor of claim 1, wherein the sensor is a thermal sensor, and the path sensor further comprises a housing configured for mounting to the conveyor to move with the conveyor, the housing having a view portion thereof that extends over at least part of the brushes and has an aperture therein to provide a field of view for the thermal sensor that encompasses the at least part of the brushes. ### Claim 3 (depends on 2) The path sensor of claim 2, wherein the controller communicates the data received from the thermal sensor to an industrial monitoring device, and the thermal sensor data is used to generate a brush temperature grid comprising a detected temperature for each of a plurality of pixels in the brush temperature grid and a corresponding visual display that differentiates the pixels based on their respective detected temperatures. ### Claim 4 (depends on 2) The path sensor of claim 2, wherein the housing is removably connected to a conveyor bracket and comprises a slot therein that is dimensioned to receive an edge of the conveyor bracket and support the view portion extending beyond the edge. ### Claim 5 (depends on 2) The path sensor of claim 2, wherein the path sensor further comprises an ozone detector, and the view portion of the housing comprises a second aperture to expose the ozone detector to the brushes when the housing is mounted to the conveyor. ### Claim 6 An industrial monitoring system for monitoring an industrial environment comprising: a plurality of multi-sensor devices (MSDs) comprising plural sensors to detect different conditions with respect to their respective positions of deployment in the industrial environment and generate corresponding sensor data, the sensor data chosen from detected distance to nearest object, vibration in one or more of three dimensional axes, orientation data in one or more of three dimensional axes, thermal camera data, video camera data, sound data, voltage measurement data corresponding to a device use in the industrial environment, temperature data, moisture data, battery level data corresponding to a battery used in the MSD, ozone data, and MSD location data; and an industrial monitoring device (IMD) having a controller, a memory device, a communication interface and a display, the communication interface connecting the IMD to each of the MSDs via their respective communication interfaces for communication using any of wireless communication and wired communication, the MSDs transmitting their sensor data to the IMD, the controller synchronizing their sensor data and generating an integrated view on a display screen comprising real-time data chosen from their sensor data; wherein, for each MSD using that MSD's sensor data, the IMD performs hypertext markup language (HTML) embedded synchronization, and generates integrated HTML video and data output for selective viewing and analysis by a user via the display. ### Claim 7 (depends on 6) The industrial monitoring system of claim 6, wherein the IMD generates a threshold alert summary for the display using the HTML video and data output for each of a plurality of the MSDs, the threshold alert summary comprising a row of icons for each of the MSDs, the icons representing respective ones of their plural sensors; in response to a user setting thresholds for at least some of sensor data collected at each MSD, the IMD determines when the thresholds are exceeded by received sensor data from the MSDs, and changes a characteristic of the corresponding icon for the sensor with sensor data that exceeds a threshold, the change in characteristic being chosen from a change in color, flashing the icon, changing intensity of icon, changing the correspond icon to a different icon. ### Claim 8 (depends on 6) The industrial monitoring system of claim 6, wherein the controller generates continuous, real-time, animated rendering of status of one or more MSDs and their data via HTML video and data output. ### Claim 9 (depends on 7) The industrial monitoring system of claim 7, wherein the IMD generates an expanded view of sensor data for the designated MDS having sensor data that exceeds its thresholds using the HTML video and data output. ### Claim 10 (depends on 7) The industrial monitoring system of claim 7, wherein the IMD generates the expanded view in response to a user selecting one of the rows. ### Claim 11 (depends on 6) The industrial monitoring system of claim 6, wherein thresholds for at least some of sensor data collected at each MSD are system configurable, and the IMD determines when the thresholds are exceeded by received sensor data from the MSDs in the HTML video and data output and generates an alert via its display. ### Claim 12 (depends on 6) The industrial monitoring system of claim 6, wherein the IMD analyzes the HTML video and data output over time to identify events wherein MSDs' thresholds were exceeded and patterns of data corresponding to failure modes to determine data signatures used for predictive maintenance of the equipment in the industrial environment being monitored via the MSDs. ### Claim 13 (depends on 6) The industrial monitoring system of claim 6, wherein the IMD plays back an output chosen from a real-time detected sound, a real-time video image, and a real-time thermal image corresponding to a selected MSD using the HTML video and data output and in response to user activation of a corresponding button provided with respect to the MSD sensor data, the button being configured by the IMD to allow a user to select a point in time in the output and view or listen to the corresponding sensor data in the HTML video and data output. ### Claim 14 (depends on 6) The industrial monitoring system of claim 6, wherein the IMD generates an MSD collected data view for the display, the MSD collected data view comprising a row for each of the MSDs, each row comprising an MSD identifier and real-time sensor data for each of a plurality of the different conditions detected by the sensors and provided via the HTML video and data output, the real-time sensor data chosen from alphanumeric values, a thumbnail view of thermal camera data, a real-time signal trace, an average of sensor data values, and a plurality of data points from the sensor data over a selected range of time relative to a designated time chosen from a user selected time and current system time. ### Claim 15 (depends on 14) The industrial monitoring system of claim 14, wherein the IMD generates an expanded view of thermal camera data corresponding to the thumbnail view of thermal camera data in response to user selection of the thumbnail view. ### Claim 16 (depends on 14) The industrial monitoring system of claim 14, wherein the IMD plays back an output chosen from a real-time detected sound, a real-time video image, and a real-time thermal image in response to user activation of a corresponding button provided with respect to the MSD collected data view. ### Claim 17 (depends on 6) The industrial monitoring system of claim 6, wherein the IMD generates a layered screen view for the display comprising sensor data corresponding to a designated MSD, the layered screen view comprising plural captured images from different fields of view relative to an item moving in the industrial environment and captured by one or more cameras chosen from a video camera and thermal camera, and overlaid alphanumeric data chosen from detected distance to nearest object, vibration in one or more of three dimensional axes, orientation data in one or more of three dimensional axes, thermal camera data, video camera data, sound data, voltage measurement data corresponding to a device use in the industrial environment, temperature data, moisture data, battery level data corresponding to a battery used in the MSD, ozone data, and MSD location data. ### Claim 18 (depends on 6) The industrial monitoring system of claim 6, wherein the industrial environment comprises a conveyor and sortation system, and the IMD imports conveyor layout and dimension data and generates a scaled graphical image of the conveyor and sortation system that comprises overlapping sensor data of the MSDs. ### Claim 19 (depends on 18) The industrial monitoring system of claim 18, wherein at least some of the MSDs are connected to the conveyor and moving when the conveyor is moving, the IMD determining locations of the moving MSDs and displaying icons representing the moving MSDs on the a scaled graphical image of the conveyor and sortation system. ### Claim 20 (depends on 19) The industrial monitoring system of claim 19, wherein the IMD uses sensor data of the MSDs and conveyor velocity to determine location of the moving MSDs. ### Claim 21 (depends on 19) The industrial monitoring system of claim 19, wherein the moving MSDs each have an associated RFID tag with unique identifier that is detected by an RFID reader deployed relative to the conveyor to determine distances between MSDs for their location determination using conveyor speed. ### Claim 22 (depends on 18) The industrial monitoring system of claim 18, wherein at least one of the MSDs is a path sensor that detects conditions of a designated moving portion of the conveyor, the conveyor portion being energized via brushes that contact a power rail, the path sensor comprising: a controller; and a sensor connected to the controller and configured to detect a condition of the brushes. ### Claim 23 (depends on 22) The industrial monitoring system of claim 22, wherein the sensor is a thermal sensor, and the path sensor further comprises a housing configured for mounting to the conveyor to move with the conveyor, the housing having a view portion thereof that extends over at least part of the brushes and has an aperture therein to provide a field of view for the thermal sensor that encompasses the at least part of the brushes. ### Claim 24 (depends on 23) The industrial monitoring system of claim 23, wherein the path sensor transmits data from the thermal sensor to the IMD, and the IMD uses the thermal sensor data to generate a brush temperature grid comprising a detected temperature for each of a plurality of pixels in the brush temperature grid, and a corresponding pixelated visual presentation on the display that differentiates the pixels based on their respective detected temperatures. ### Claim 25 (depends on 24) The industrial monitoring system of claim 24, wherein thresholds for temperatures of the brushes are system configurable, and the IMD generates an alert detected temperatures indicated in the pixels that correspond to the brushes exceeds their thresholds. ### Claim 26 (depends on 18) The industrial monitoring system of claim 18, wherein at least one of the MSDs is a package sensor that detects conditions experienced by a package transported within the conveyor and sortation system, the plural sensors deployed in the package sensor to detect different conditions being chosen detected distance to nearest object, vibration in one or more of three dimensional axes, orientation data in one or more of three dimensional axes, thermal camera data, video camera data, and sound data, the plural sensors being reconfigurable for mounting in packages having different form factors and chosen from tote, tray, box, crate, carrier and cart.