Factory data generation with intelligent labelling

An apparatus for subtask identification and analyzing automatically-generated labels associated with identified subtasks to identify an optimal subtask ordering associated with performing at least one task, comprising:

• a display,
• a memory;
• and
• at least one processor coupled to the memory and, based at least in part on information stored in the memory, the at least one processor is configured to:
• receive video data associated with the at least one task performed by at least one worker, wherein the at least one task is a pre-defined task associated with an industrial process;
• execute, based on the received video data, a human action recognition program to identify at least one action associated with the at least one task;
• execute, based on the received video data, an object detection program to identify at least one object associated with the at least one task;
• identify, based on a combination of the identified at least one action and the identified at least one object associated with the at least one task, at least one subtask of the at least one task;
• generate at least one label for the identified at least one subtask based on at least a first label associated with the at least one identified action from a first set of existing labels for identified actions and at least a second label associated with the identified at least one object from a second set of existing labels for identified objects, wherein the first set of existing labels and the second set of existing labels are stored in the memory;
• calculate a similarity score between the generated at least one label and a task description provided for the pre-defined task;
• determine whether to store or discard the generated at least one label for further analysis based on the calculated similarity score;
• analyze a set of generated labels for subtasks of the at least one task for each of a plurality of workers performing the at least one task to determine the optimal subtask ordering associated with performing the at least one task;
• and
• present, via the display, a visual representation of the optimal subtask ordering.

A method for automatically identifying subtasks, from video data associated with a task performed by one or more workers, and generating labels associated with identified subtasks and the video data, and analyzing automatically-generated labels associated with identified subtasks to identify an optimal subtask ordering associated with performing at least one task, comprising:

• at least one processor coupled to at least one memory and, based at least in part on information stored in the at least one memory, the at least one processor is configured to perform the steps of:
• receiving video data associated with the at least one task performed by at least one worker, wherein the at least one task is a pre-defined task associated with an industrial process;
• executing, based on the received video data, a human action recognition program to identify at least one action associated with the at least one task;
• executing, based on the received video data, an object detection program to identify at least one object associated with the at least one task;
• identifying, based on a combination of the identified at least one action and the identified at least one object associated with the at least one task, at least one subtask of the at least one task;
• generating at least one label for the identified at least one subtask based on at least a first label associated with the at least one identified action from a first set of existing labels for identified actions and at least a second label associated with the identified at least one object from a second set of existing labels for identified objects, wherein the first set of existing labels and the second set of existing labels are stored in the at least one memory associated with the human action recognition program and the object detection program;
• calculating a similarity score between the generated at least one label and a task description provided for the pre-defined task;
• determining whether to store or discard the generated at least one label for further analysis based on the calculated similarity score;
• analyzing a set of generated labels for subtasks of the at least one task for each of a plurality of workers performing the at least one task to determine the optimal subtask ordering associated with performing the at least one task;
• and
• presenting, via a display, a visual representation of the optimal subtask ordering.