Robotic packing of unknown objects

An autonomous system configured to pick and place objects into containers, the autonomous system comprising:

• a robot defining an end effector configured to grasp a first object within a workspace, the first object defining a bottom end and a top end opposite the bottom end along a transverse direction;
• a first camera configured to capture a first image of the first object along the transverse direction while the robot grasps the first object, the first image including the bottom end;
• a second camera configured to capture a second image of a container within the workspace;
• and
• a memory storing instructions that, when executed by the one or more processors, cause the autonomous system to:
• based on the second image, generate a heightmap of the container, the heightmap indicating respective distances defined by the container and any objects within the container along the transverse direction;
• and
• based on the heightmap and the first image, perform an optimization so as to determine an optimal pose within the container for the first object, the optimal pose defining a location within the container and an orientation of the first object.