Image-based automated ergonomic risk root cause and solution identification system and method

A computer-implemented method performed by a computing server system, the computer-implemented method comprising:

• obtaining, by a processor of the computing server system, an image of a worker performing a work task;
• encoding the image to generate an embedding vector;
• incorporating a deep learning based model to process the embedding vector to generate a first sentence indicating a root cause of an ergonomic risk identified in the image and a second sentence indicating a solution to the ergonomic risk identified in the image;
• and
• establishing a dataset to train the deep learning based model by:
• collecting samples of image-caption data, the image-caption data including a plurality of images and information relating to ergonomic problems identified in the plurality of images and corresponding solutions to the ergonomic problems,
• categorizing the samples of image-caption data into a plurality of ergonomic problem-solution pairs,
• identifying, from the samples, training images associated with each of the plurality of ergonomic problem-solution pairs,
• sampling the plurality of ergonomic problem-solution pairs in connection with the training images,
• establishing the dataset based at least upon sampling results of the plurality of ergonomic problem-solution pairs in connection with the training images, and
• training the deep learning based model using the dataset.

A non-transitory computer readable medium storing machine executable instructions for a computing server system, the machine executable instructions being configured for:

• obtaining, by a processor of the computing server system, an image of a worker performing a work task;
• encoding the image to generate an embedding vector;
• incorporating a deep learning based model to process the embedding vector to generate a first sentence indicating a root cause of an ergonomic risk identified in the image and a second sentence indicating a solution to the ergonomic risk identified in the image;
• and
• establishing a dataset to train the deep learning based model by:
• collecting samples of image-caption data, the image-caption data including a plurality of images and information relating to ergonomic problems identified in the plurality of images and corresponding solutions to the ergonomic problems,
• categorizing the samples of image-caption data into a plurality of ergonomic problem-solution pairs,
• identifying, from the samples, training images associated with each of the plurality of ergonomic problem-solution pairs,
• sampling the plurality of ergonomic problem-solution pairs in connection with the training images,
• establishing the dataset based at least upon sampling results of the plurality of ergonomic problem-solution pairs in connection with the training images, and
• training the deep learning based model using the dataset.