Collision detection for object rearrangement using a 3d scene representation

A system, comprising:

• a non-transitory memory storage comprising instructions;
• and
• one or more processors in communication with the memory, wherein the one or more processors execute the instructions to:
• process at least one image of a scene having a plurality of objects to learn a three-dimensional representation of the scene;
• perform collision detection for at least a portion of the scene, using the three-dimensional representation of the scene, to sample a target object from the plurality of objects;
• predict a plurality of waypoints in the scene for a rearrangement task corresponding to the target object, using the three-dimensional representation of the scene.