Robot interaction with human co-workers

A system for controlling a robot for object avoidance, the system comprising:

• a processor;
• and
• a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the system to:
• detect a type and a location of an object based on a camera image of the object, the image having a reference frame, by determining a six degree-of-freedom (6DOF) pose using pixels of the object, including an x-position, y-position, z-position, roll, pitch, and yaw of the object, represented in the reference frame of the image of the object;
• predict motion of the object based on the detected type of the object, the detected location of the object, and a physics-based dynamics model of the object-, the predicting motion comprising:
• selecting equations to include in the physics-based dynamics model of the object from a set of plausible models by matching the detected type of the object to an element in a library of models;
• and
• performing a simulation using both:
• (i) the physics-based dynamics model of the object, including the selected equations, and (ii) a recursive Bayesian estimator configured to employ the physics-based dynamics model of the object, including the selected equations, as a system model to update the recursive Bayesian estimator, the performing the simulation providing the predicted motion of the object;
• and
• generate a motion plan for a robot that avoids having the robot collide with the object based on:
• the predicted motion of the object;
• determining a transformation between the reference frame of the image and a reference frame of the robot, the determined transformation between the reference frame of the image and the reference frame of the robot enabling conversion from locations in the image to locations in relation to the robot;
• and
• determining a time varying model of the object based on the predicted motion of the object.

A non-transitory computer program product for controlling a robot for object avoidance, the computer program product comprising a computer-readable medium with computer code instructions stored thereon, the computer code instructions being configured, when executed by a processor, to cause an apparatus associated with the processor to:

• detect a type and a location of an object based on a camera image of the object, the image having a reference frame, by determining a six degree-of-freedom (6DOF) pose using pixels of the object, including an x-position, y-position, z-position, roll, pitch, and yaw of the object, represented in the reference frame of the image of the object;
• predict motion of the object based on the detected type of the object, the detected location of the object, and a physics-based dynamics model of the object-, the predicting motion comprising:
• selecting equations to include in the physics-based dynamics model of the object from a set of plausible models by matching the detected type of the object to an element in a library of models;
• and
• performing a simulation using both:
• (i) the physics-based dynamics model of the object, including the selected equations, and (ii) a recursive Bayesian estimator configured to employ the physics-based dynamics model of the object, including the selected equations, as a system model to update the recursive Bayesian estimator, the performing the simulation providing the predicted motion of the object;
• and
• generate a motion plan for a robot that avoids having the robot collide with the object based on:
• the predicted motion of the object;
• determining a transformation between the reference frame of the image and a reference frame of the robot, the determined transformation between the reference frame of the image and the reference frame of the robot enabling conversion from locations in the image to locations in relation to the robot;
• and
• determining a time varying model of the object based on the predicted motion of the object.