Systems and methods automatic anomaly detection in mixed human-robot manufacturing processes

A process control method for detecting an anomaly in an execution of a task in a sequence of tasks in mixed human-robot processes, comprising:

• acquiring human-worker (HW) signals and robot operational signals from sensors;
• extracting from the HW signals, a task completion time, measurements relating to a state of the HW and a next sequenced task, and input into a Human Performance (HP) model, to obtain a state of the HW based on previously learned boundaries of the state of the HW, the state of the HW is then inputted into a Human-Robot Interaction (HRI) model, to determine a classification of an anomaly or no anomaly, wherein the HRI model is previously configured to have learned mappings between the different states of the HW and optimal robot actions;
• updating the HRI model with the robot operation signals, the HW signals and the classified anomaly, then determine a control action of a robot interacting with the HW or a type of an anomaly alarm using the updated HRI model and the classified anomaly;
• and
• outputting the control action of the robot to change a robot action or output the type of the anomaly alarm to a management system of the mixed human-robot processes, based on the updated HRI model and the classified anomaly, wherein the steps are implemented by a hardware processor connected to a memory.

A non-transitory computer readable storage medium embodied thereon a program executable by a computer for performing a process control method, the process control method for detecting an anomaly in an execution of a task in a sequence of tasks in mixed human-robot processes, the process control method comprising steps of:

• acquiring human worker (HW) signals and robot operational signals from sensors;
• extracting from the HW signals, a task completion time, measurements relating to a state of the HW and a next sequenced task, and input into a Human Performance (HP) model, to obtain a state of the HW performance based on previously learned boundaries of the state of the HW, the state of the HW performance is then inputted into a Human-Robot Interaction (HRI) model, to determine a classification of anomaly or no anomaly, wherein the HRI model is previously configured to have learned mappings between the different states of the HW and optimal robot actions;
• updating the HRI model with the robot operation signals, the HW signals and the classified anomaly, then determine a control action of a robot interacting with the HW or a type of an anomaly alarm using the updated HRI model and the classified anomaly;
• and
• outputting the control action of the robot to change a robot action or output the type of the anomaly alarm to a management system of the mixed human-robot processes, based on the updated HRI model and the classified anomaly, wherein the steps of the process control method are implemented using a hardware processor connected to a memory.