US 10,896,331 B2 · Monitoring activities in industrial sites

Abstract

A computer-implemented method and system for monitoring activities in industrial sites using data models is presented. In an embodiment, a method comprises: using a computing device, receiving a plurality of data inputs from a plurality of data input devices in an industrial site; using the computing device, obtaining access to a data model that has been trained on training data for the industrial site and that is programmed to detect one or more anomalies associated with workers or equipment at the site; using the computing device, applying the plurality of data inputs to the data model and evaluating the data model with the data inputs to result in receiving output data specifying whether the plurality of data inputs indicates one or more anomalies occurring in the industrial site; using the computing device, in response to determining that the output data indicates a set of anomalies, of the one or more anomalies: generating a first set of notifications that corresponds to the set of anomalies occurring in the industrial site; for each first anomaly of the first set of anomalies: generating a first notification that includes a description of a first anomaly of the first set of anomalies; determining one or more first recipients of the first notification; transmitting the first notification to the one or more first recipients.

Drawings

Claims21

Dependent claims

A computer-implemented method comprising:

using a computing device, receiving a plurality of data inputs from a plurality of data input devices in an industrial site;
identifying, in the plurality of data inputs, a plurality of digital images depicting workers working on the industrial site;
detecting a plurality of color-coded objects depicted in the plurality of digital images;
wherein detecting a plurality of color-coded objects in the plurality of digital images includes one or more of:
detecting colors of hard hats indicating worker trades and depicted in the plurality of digital images, detecting markers attached to hard hats indicating worker trades and depicted in the plurality of digital images, or detecting any other color-coded indicia embedded on hard hats indicating worker trades and depicted in the plurality of digital images;
based on the color-coded objects, determining a plurality of worker counts of workers that belong to different trades;
using the computing device, obtaining access to a data model that has been trained on training data for the industrial site and that is programmed to detect one or more anomalies associated with workers or equipment at the industrial site;
using the computing device, applying the plurality of data inputs and the plurality of worker counts to the data model and evaluating the data model with the data inputs to result in receiving output data specifying whether the plurality of data inputs and the plurality of worker counts indicate one or more anomalies occurring in the industrial site;
using the computing device, in response to determining that the output data indicates a first set of anomalies, of the one or more anomalies:
generating a first set of notifications that corresponds to the first set of anomalies occurring in the industrial site;
for each first anomaly of the first set of anomalies:
generating a first notification that includes a description of a first anomaly of the first set of anomalies;
determining one or more first recipients of the first notification;
transmitting the first notification to the one or more first recipients.

11 more elements

A computer system comprising one or more processors, one or more non-transitory digital storage media coupled to the processors and storing one or more sequences of program instructions which, when executed using the one or more processors, cause the one or more processors to perform:

using a computing device, receiving a plurality of data inputs from a plurality of data input devices in an industrial site;
identifying, in the plurality of data inputs, a plurality of digital images depicting workers working on the industrial site;
detecting a plurality of color-coded objects depicted in the plurality of digital images;
wherein detecting a plurality of color-coded objects in the plurality of digital images includes one or more of:
detecting colors of hard hats indicating worker trades and depicted in the plurality of digital images, detecting markers attached to hard hats indicating worker trades and depicted in the plurality of digital images, or detecting any other color-coded indicia embedded on hard hats indicating worker trades and depicted in the plurality of digital images;
based on the color-coded objects, determining a plurality of worker counts of workers that belong to different trades;
using the computing device, obtaining access to a data model that has been trained on training data for the industrial site and that is programmed to detect one or more anomalies associated with workers or equipment at the industrial site;
using the computing device, applying the plurality of data inputs and the plurality of worker counts to the data model and evaluating the data model with the data inputs to result in receiving output data specifying whether the plurality of data inputs and the plurality of worker counts indicate one or more anomalies occurring in the industrial site;
using the computing device, in response to determining that the output data indicates a first set of anomalies, of the one or more anomalies:
generating a first set of notifications that corresponds to the first set of anomalies occurring in the industrial site;
for each first anomaly of the first set of anomalies:
generating a first notification that includes a description of a first anomaly of the first set of anomalies;
determining one or more first recipients of the first notification;
transmitting the first notification to the one or more first recipients.

11 more elements

One or more non-transitory digital storage media storing one or more sequences of program instructions which, when executed using one or more processors, cause the one or more processors to perform:

using a computing device, receiving a plurality of data inputs from a plurality of data input devices in an industrial site;
identifying, in the plurality of data inputs, a plurality of digital images depicting workers working on the industrial site;
detecting a plurality of color-coded objects depicted in the plurality of digital images;
wherein detecting a plurality of color-coded objects in the plurality of digital images includes one or more of:
detecting colors of hard hats indicating worker trades and depicted in the plurality of digital images, detecting markers attached to hard hats indicating worker trades and depicted in the plurality of digital images, or detecting any other color-coded indicia embedded on hard hats indicating worker trades and depicted in the plurality of digital images;
based on the color-coded objects, determining a plurality of worker counts of workers that belong to different trades;
using the computing device, obtaining access to a data model that has been trained on training data for the industrial site and that is programmed to detect one or more anomalies associated with workers or equipment at the industrial site;
using the computing device, applying the plurality of data inputs and the plurality of worker counts to the data model and evaluating the data model with the data inputs to result in receiving output data specifying whether the plurality of data inputs and the plurality of worker counts indicate one or more anomalies occurring in the industrial site;
using the computing device, in response to determining that the output data indicates a first set of anomalies, of the one or more anomalies:
generating a first set of notifications that corresponds to the first set of anomalies occurring in the industrial site;
for each first anomaly of the first set of anomalies:
generating a first notification that includes a description of a first anomaly of the first set of anomalies;
determining one or more first recipients of the first notification;
transmitting the first notification to the one or more first recipients.

11 more elements

Dependencies are inferred from the claim text.

Description12,771 words

Summary396 words

#One technical field of the is methods to monitor and track persons, physical assets, and deliveries of parts and materials in industrial sites. Another technical field is computer-implemented machine vision. Yet another technical field artificial intelligence-based processing.

#The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.

#Managing large industrial sites often includes considering safety, efficiency, and accountability of the persons present on the sites. This may also include considering efficiency and accountability of the activities taking place on the sites. In the context of an industrial site, the term safety refers to the safety of workers and pedestrians present on or near the industrial site; the term efficiency refers to the optimization of resources such as equipment assets; and the term accountability refers to the ability to validate the constructed product against the specification of the industrial project, and the accuracy in billing for the work done.

Brief description of the drawings12 figures

#In the drawings:

#FIG. 1 illustrates an example computer system that may be used to implement an embodiment.

#FIG. 2 depicts an example map showing an industrial site and locations of video cameras implemented in the site.

Detailed description12,173 words

#In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the method described herein. It will be apparent, however, that the present approach may be practiced without these specific details. In some instances, well-known structures and devices are shown in a block diagram form to avoid unnecessarily obscuring the present approach.

#1. OVERVIEW

#2. EXAMPLE SYSTEM ARCHITECTURE

Facts

Application

16/129,699Filed 12 Sep 2018 · us provisional application

Priority

29 Jan 2018

Published

US 2019/0236370 A11 Aug 2019

Granted

19 Jan 2021

Term ends

12 Sep 2038

Estimated · how

20 years from earliest known nonprovisional domestic filing plus recorded patent term adjustment. Maintenance-fee lapses, terminal disclaimers and statutory extensions have not been checked. The continuity chain may be incomplete; this is not an enforceability determination.

Classification7 codes

G06V 20/40
Scenes; Scene-specific elements: in video content
G05B 19/41875
Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible…:…
G06Q 10/0633
Operations research, analysis or management: workflow analysis
G06V 20/52
Context or environment of the image: surveillance or monitoring of activities, e.g. for recognising suspicious objects

Timeline

29 Jan 2018Priority claimedParent application 62/623,392
12 Sep 2018Application filedUS 16/129,699
13 Sep 2018Assigned to Indus.Ai, Inc.assignment of assignor's interest
18 Sep 2018Assigned to Indus.Ai Incassignment of assignor's interest
1 Aug 2019PublishedUS 2019/0236370 A1
19 Jan 2021GrantedUS 10,896,331 B2
19 Apr 2021Assigned to Indus.Ai Inc.assignment of assignor's interest
17 Dec 2021Assigned to Procore Technologies, Inc.assignment of assignor's interest
12 Oct 2026TodayWithin its term
12 Sep 2038Term endsEstimated

Monitoring activities in industrial sites

Life of this patent

Monitoring activities in industrial sites

Life of this patent

Life of this patent

Abstract

Drawings

Drawings

Claims21

Description12,771 words

Drawings

Classification7 codes

Life of this patent

Description12,771 words