Machine learning system for technical knowledge capture

A method for capturing knowledge for performing a task, the method comprising:

• obtaining, by a video input device, video data of a first user performing a plurality of steps for performing the task;
• obtaining, by an audio input device, audio data describing performance of the plurality of steps for performing the task;
• generating, by one or more sensors, sensor data during performance of the plurality of steps for performing the task;
• correlating, by a computation engine, at least two of the video data, the audio data, and the sensor data to identify at least a portion of each of the at least two of the video data, the audio data, and the sensor data that depicts a same step of the plurality of steps;
• processing, by the computation engine, the identified at least a portion of each of the at least two of the video data, the audio data, and the sensor data to update a domain model defining the plurality of steps for performing the task;
• applying, by a training unit, the updated domain model to generate training information for performing the plurality of steps for performing the task;
• and
• outputting, by an output device, the training information for use in training a second user to perform the plurality of steps for performing the task.

A non-transitory, computer-readable medium comprising instructions that, when executed, are configured to cause processing circuitry to:

• obtain video data of a first user performing a plurality of steps for performing a task;
• obtain audio data describing performance of the plurality of steps for performing the task;
• generate sensor data during performance of the plurality of steps for performing the task;
• correlate at least two of the video data, the audio data, and the sensor data to identify at least a portion of each of the at least two of the video data, the audio data, and the sensor data that depicts a same step of the plurality of steps;
• process the identified at least a portion of each of the at least two of the video data, the audio data, and the sensor data to update the domain model defining the plurality of steps for performing the task;
• apply the updated domain model to generate training information for performing the plurality of steps for performing the task;
• and
• output the training information for use in training a second user to perform the plurality of steps for performing the task.