State estimation using geometric data and vision system for palletizing

A method to control a robot, comprising:

• receiving, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on the pallet or in the other receptacle;
• controlling, by one or more processors, a robotic arm to place a first set of items on or in, or remove the first set of items from, the pallet or the other receptacle;
• updating a geometric model based on the first set of items placed on or in, or removed from, the pallet or the other receptacle;
• generating an estimated state of the pallet or the other receptacle and one or more items stacked on the pallet or in the other receptacle, wherein the estimated state is determined based on a combination of (i) the geometric model and (ii) a noise according to a predefined noise profile;
• and
• using the estimated state to generate or update a plan to control the robotic arm to place a second set of items on or in, or remove the second set of items from, the pallet or the other receptacle.

A non-transitory computer readable mediums storing thereon a computer program product, comprising computer instructions executed by a processor for:

• receiving, from one or more sensors deployed in a workspace, sensor data indicative of a current state of the workspace, the workspace comprising a pallet or other receptacle and a plurality of items stacked on the pallet or in the other receptacle;
• controlling, by one or more processors, a robotic arm to place a first set of items on or in, or remove the first set of items from, the pallet or the other receptacle;
• updating a geometric model based on the first set of items placed on or in, or removed from, the pallet or the other receptacle;
• generating an estimated state of the pallet or the other receptacle and one or more items stacked on the pallet or in the other receptacle, wherein the estimated state is determined based on a combination of (i) the geometric model and (ii) a noise according to a predefined noise profile;
• and
• using the estimated state to generate or update a plan to control the robotic arm to place a second set of items on or in, or remove the second set of items from, the pallet or the other receptacle.