Behavior-based standard operating procedure detection

A system, comprising:

• one or more processors;
• and
• a computer-readable storage device coupled to the one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for gesture-based standard operating procedure (SOP) detection, the operations comprising:
• receiving a first set of images included in a first video captured by a camera that monitors a human performing a task;
• processing the first set of images using a first machine learning (ML) model to determine whether the first set of images depicts a gesture that is included in a predefined set of gestures;
• in response to determining that the first set of images depicts a gesture included in a predefined set of gestures, processing a second set of images included in the first video using a second ML model to determine a first gesture type of the gesture;
• comparing the first gesture type with a first expected gesture type to determine whether performance of the task conforms to a standard operating procedure (SOP) for the task;
• and
• providing feedback representative of a comparison result in a user interface.

Computer-readable storage media coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for behavior-based standard operating procedure (SOP) detection, the operations comprising:

• receiving a first set of images included in a first video captured by a camera that monitors a human performing a task;
• processing the first set of images using a first machine learning (ML) model to determine whether the first set of images depicts a gesture that is included in a predefined set of gestures;
• in response to determining that the first set of images depicts a gesture included in a predefined set of gestures, processing a second set of images included in the first video using a second ML model to determine a first gesture type of the gesture;
• comparing the first gesture type with a first expected gesture type to determine whether performance of the task conforms to a standard operating procedure (SOP) for the task;
• and
• providing feedback representative of a comparison result in a user interface.