Machine learning methods and apparatus for automated robotic placement of secured object in appropriate location

A robot, comprising:

• a vision component;
• actuators;
• an end effector;
• memory storing:
• instructions, a trained classification model, and a trained placement model;
• one or more processors, executing the instructions, to perform a method comprising:
• identifying one or more object images, from the vision component, that each capture an object in an environment of the robot;
• generating a target placement input that indicates at least one characteristic of a placement location for the object, wherein generating the target placement input is based on classification output generated by processing the one or more object images using the trained classification model;
• while the end effector of the robot is securing the object:
• identifying a current image, from the vision component, that captures the end effector and at least part of the environment;
• generating candidate end effector actions that each define corresponding motion to move the end effector;
• for each candidate end effector action of the candidate end effector actions:
• processing, using the trained placement model, the target placement input, the current image, and the candidate end effector action, and
• generating, based on the processing, a prediction of successful placement of the object with application of the motion defined by the corresponding candidate end effector action;
• generating control commands that conform to the candidate end effector action that, when processed using the trained placement model, resulted in the prediction that is most indicative of successful placement of the object;
• and
• providing the control commands to one or more of the actuators of the robot to cause the end effector to move while continuing to secure the object;
• determining to cause the end effector to release the object after providing the control commands;
• and
• in response to determining to cause the end effector to release the object, providing one or more additional control commands to cause the end effector to release the object.