Robotic meal-assembly systems and robotic methods for real-time object pose estimation of high-resemblance random food items

A robotic method for automatic kitting of items comprising:

• imaging kitted items to generate first image data;
• determining whether each item desired to be in the kitted items is present or absent in the kitted items in response to the first image data, comprising determining a type of each item in the kitted items and a position of that item in the kitted items in response to the first image data;
• generating data based on whether an item desired to be in the kitted items is absent from the kitted items in response to the type and the position of each item present in the first image data;
• generating robotic control instructions in response to the item desired to be in the kitted items being absent from the kitted items;
• and
• providing the robotic control instructions to a robotic device, such that the robotic device controls movement of an arm and a robotic gripper of the robotic device to pick an item corresponding to the item desired to be in the kitted items which is absent from the kitted items and place the picked item into the kitted items.

A non-transitory computer readable medium comprising instructions for automatic kitting of items in a robotic system, the instructions causing a controller in the robotic system to:

• image kitted items to generate first image data;
• determine whether each item desired to be in the kitted items is present or absent in the kitted items in response to the first image data, comprising determining a type of each item in the kitted items and a position of that item in the kitted items in response to the first image data;
• generate data based on whether an item desired to be in the kitted items is absent from the kitted items in response to the type and the position of each item present in the first image data;
• generate robotic control instructions in response to the item desired to be in the kitted items being absent from the kitted items;
• and
• provide the robotic control instructions to a robotic device, such that the robot device controls movement of an arm and a robotic gripper of the robotic device, wherein the robotic control instructions comprise one or more instructions to pick an item corresponding to the item desired to be in the kitted items which is absent from the kitted items and place the picked item into the kitted items.