Systems and methods for robotic picking and perturbation

A method of operating an apparatus for picking items in a bin, the method comprising:

• collecting visual information related to a current state of one or more items in the bin;
• inputting the visual information into one or more machine learning algorithms configured to identify picking strategies;
• performing a first pick up attempt for an item of the one or more items in the bin according to a picking strategy identified by the one or more machine learning algorithms configured to identify picking strategies;
• determining that the first pick up attempt failed;
• in response to determining that the first pick up attempt failed, inputting updated visual information into one or more different machine learning algorithms configured to identify a perturbation sequence based at least in part on an arrangement of the one or more items in the bin, wherein the updated visual information is related to a updated state of the one or more items in the bin after the first pick up attempt;
• repositioning the one or more items in the bin with compressed air from at least one end effector according to the perturbation sequences identified by the one or more different machine learning algorithms, wherein the perturbation sequence comprises movement in a plurality of directions relative to the bin;
• collecting new visual information related to a new state of the one or more items in the bin;
• inputting the new visual information into the one or more machine learning algorithms configured to identify picking strategies;
• and
• performing a second pick up attempt for the item according to a new picking strategy identified by the one or more machine learning algorithms configured to identify picking strategies.

A system comprising:

• one or more computer readable storage media;
• one or more processing systems operatively coupled to the one or more computer readable storage media;
• and
• program instructions, stored on the one or more computer readable storage media, for directing a robotic device to pick and perturb items, wherein the program instructions, when read and executed by the one or more processing systems, direct the robotic device to:
• collect visual information related to a current state of one or more items in an area;
• input the visual information into one or more machine learning algorithms configured to identify picking strategies;
• perform a first pick up attempt for an item of the one or more items in the area according to a picking strategy identified by the one or more machine learning algorithms configured to identify picking strategies;
• in response to determining that the first pick up attempt failed, input updated visual information into one or more different machine learning algorithms configured to identify a perturbation sequence based at least in part on an arrangement of the one or more items in the area, wherein the updated visual information relates to an updated state of the one or more items in the area after the first pick up attempt;
• reposition the one or more items in the area using compressed air from at least one end effector according to the perturbation sequence identified by the one or more different machine learning algorithms, wherein the perturbation sequence comprises movement in a plurality of directions relative to the area;
• collect new visual information related to a new state of the one or more items in the area;
• input the new visual information into the one or more machine learning algorithms configured to identify picking strategies;
• and
• perform a second pick up attempt for the item according to a new picking strategy identified by the one or more machine learning algorithms configured to identify picking strategies.