Imitation Learning in a Manufacturing Environment

A method for training a control system, comprising:

• receiving, by a computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system;
• and
• based on the initial teacher policy, training, by the computing system, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
• causing the control system to mimic the movements of the human operator while performing the task,
• monitoring the movements of the control system using sensors,
• generating an output trajectory of the control system performing the task based on the monitored movements,
• determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
• receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
• causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example, monitoring additional movements of the control system using the sensors,
• generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
• determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.

A non-transitory computer readable medium comprising one or more sequences of instructions, which, when executed by a processor, causes a computing system to perform operations comprising:

• receiving, by the computing system, an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system of the computing system;
• and
• based on the initial teacher policy, training, by the computing system, the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
• causing the control system to mimic the movements of the human operator while performing the task,
• monitoring the movements of the control system using sensors,
• generating an output trajectory of the control system performing the task based on the monitored movements,
• determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
• receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
• causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example,
• monitoring additional movements of the control system using the sensors,
• generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
• determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.

A system comprising:

• a processor;
• and
• a memory having programming instructions stored thereon, which, when executed by the processor, causes the system to perform operations comprising:
• receiving an initial teacher policy based on a trajectory example generated by a human operator in a first action space, the trajectory example captured using one or more sensors monitoring movements of the human operator, the trajectory example comprising trajectory information of the human operator while performing a task to be learned by a control system;
• and
• based on the initial teacher policy, training the control system in a second action space having a lower dimension than the first action space to mimic the movements of the human operator performing the task, the training comprising:
• causing the control system to mimic the movements of the human operator while performing the task,
• monitoring the movements of the control system using sensors,
• generating an output trajectory of the control system performing the task based on the monitored movements,
• determining that the output trajectory of the control system deviates from the trajectory information of the human operator while performing the task by a threshold amount,
• receiving an updated teacher policy based on an updated trajectory example generated by the human operator, the updated trajectory example comprising updated trajectory information of the human operator while performing the task to be learned by the control system, the updated trajectory example comprising at least one movement that differs from the trajectory example,
• causing the control system to mimic the movements of the human operator corresponding to the updated trajectory example,
• monitoring additional movements of the control system using the sensors, generating an updated output trajectory of the control system performing the task based on the monitored additional movements, and
• determining that the updated output trajectory of the control system deviates from the updated trajectory information of the human operator while performing the task by less than the threshold amount.