System and method for mapping risks in a warehouse environment

A computer implemented system for identifying and mitigating risks in a warehouse environment, the computer implemented system comprising:

• a plurality of video sensors configured to obtain a plurality of real time video frames over a pre-defined period of time;
• a central processing unit coupled to the plurality of video sensors, wherein the central processing unit is configured with:
• a risk discovery unit configured to:
• detect a current risk subject including at least one of a person, a pallet and a package in the obtained plurality of real time video frames based on activation of one or more pre-defined trigger conditions, wherein the risk subject is one of a person, pallets and packages visible in the plurality of video frames, and wherein the detected risk subject is a subject having associated therewith at least one of a predefined risk known beforehand and a heuristic risk previously unknown;
• store the obtained plurality of real time video frames in a memory;
• and
• determine a location of the current risk subject detected in the obtained plurality of real time video frames;
• a trajectory computation unit comprising:
• a previous state database configured to store a previous state data derived from observing a previously detected risk subject from a first video frame obtained from the plurality of real time video frames;
• a Kalman filter unit configured to retrieve detection measurement data calculated from the determined location of the one or more risk subjects, and a previous state data of the previously detected risk subject from the previous state database, and calculate therefrom a predicted state data and an actual measurement data of the current risk subject;
• a tracking database communicatively coupled to the Kalman filter unit and configured to store tracklet data associated with previously detected risk subjects, wherein the tracklet data comprises a previous appearance data of the previously detected risk subjects;
• a matcher unit communicatively coupled to the Kalman filter unit and configured to:
• (i) calculate a first distance between the actual measurement data and the predicted state data and implement a thresholding function on the first distance to form a state indicator data;
• and (ii) calculate a second distance between a detection appearance data of the current risk subject and the previous appearance data of the tracklet data and implement a thresholding function on the second distance to form an appearance indicator data;
• and
• a Hungarian unit communicatively coupled to the matcher unit and configured to calculate an association data based on a product of the state indicator data and the appearance indicator data, and match a current detection of the risk subject to the tracklet data when the association data equals a pre-defined value to form a set of first matching indices and second matching indices linking the current detection of the risk subject to a corresponding tracklet vector in the tracking database;
• a human pose estimation unit configured to estimate a multi-point skeletal pose of the current risk subject from the plurality of real time video frames and classify one or more actions performed by the current risk subject based on the estimated skeletal pose over a pre-defined time window;
• and
• a pattern analysis unit communicably coupled to the trajectory computation unit and the human pose estimation unit and configured to process the determined location of the current risk subject to detect patterns or movements and activities undertaken by one or more risk subjects;
• and
• wherein the central processing unit is further configured to provide actionable insights to warehouse managers to take one or more remedial actions for preventing, or at least minimizing, an effect associated with risk factors identified for the one or more risk subjects and for addressing a cause thereof by generating and providing to the warehouse managers at least one of:
• a redesigned physical and logistics plan of the warehouse;
• a redefined manipulation procedure to be performed by, or on, the risk subjects;
• an improved packing and stacking criteria over an existing packing and stacking criteria for the risk subject;
• alternative order pickers routes that are different from existing order pickers routes;
• and
• an enhanced warehouse environment and operator monitoring protocol different from an existing environmental and operator monitoring protocol.

A non-transitory computer-readable medium for identifying and mitigating risks in a warehouse environment, the non-transitory computer-readable medium comprising a set of instructions, the set of instructions when executed by a processor causes the processor to:

• detect a current risk subject including at least one of a person, a pallet and a package in a plurality of real time video frames, and wherein the detected risk subject is a subject having associated therewith at least one of a predefined risk known beforehand and a heuristic risk previously unknown;
• store the plurality of real time video frames in a memory;
• determine a location of the current risk subject detected in the plurality of real time video frames;
• crop the plurality of real time video frames based on calculated detection measurement data derived from the determined location of the current risk subject to extract detection appearance data of the current risk subject;
• retrieve, using a Kalman filter unit, the detection measurement data and a previous state data of a previously detected risk subject stored in a previous state database, and calculate therefrom a predicted state data and an actual measurement data for the current risk subject;
• calculate, using a matcher unit, a first distance between the actual measurement data and the predicted state data and apply a thresholding function on the first distance to form a state indicator data, and calculate a second distance between the detection appearance data of the current risk subject and a previous appearance data stored in tracklet data of a tracking database and apply a thresholding function on the second distance to form an appearance indicator data;
• calculate, using a Hungarian unit, an association data based on a product of the state indicator data and the appearance indicator data, and match a current detection of the risk subject to the tracklet data when the association data equals a pre-defined value to form a set of first matching indices and second matching indices linking the current detection of the risk subject to a corresponding tracklet vector in the tracking database;
• estimate a multi-point skeletal pose of the current risk subject using a human pose estimation unit, and classify one or more actions performed by the current risk subject based on the estimated skeletal pose over a pre-defined time window;
• process, using a pattern analysis unit communicably coupled to the trajectory computation unit and the human pose estimation unit, the determined location of the current risk subject to detect patterns or movements and activities undertaken by one or more risk subjects;
• and
• provide actionable insights to warehouse managers to take one or more remedial actions for preventing, or at least minimizing, an effect associated with risk factors identified for the one or more risk subjects and for addressing a cause thereof by generating and providing to the warehouse managers at least one of:
• a redesigned physical and logistics plan of the warehouse;
• a redefined manipulation procedure to be performed by, or on, the risk subjects;
• an improved packing and stacking criteria over an existing packing and stacking criteria for the risk subject;
• alternative order pickers routes that are different from existing order pickers routes;
• and
• an enhanced warehouse environment and operator monitoring protocol different from an existing environmental and operator monitoring protocol.