Excavation learning for rigid objects in clutter

A system for autonomous excavation learning and planning comprising:

• one or more processors;
• and
• a non-transitory computer-readable medium or media comprising one or more sets of instructions which, when executed by at least one of the one or more processors, causes steps to be performed comprising:
• tiling an excavation trajectory, presented by multiple trajectory parameters, across a voxel-grid dimension to obtain tiled trajectory parameter voxel-grids;
• concatenating the tiled trajectory parameter voxel-grids with a voxel-grid representation of an excavation scene to generate an input voxel-grid of the excavation instance that defines a pair of the excavation trajectory and the voxel-grid representation of the excavation scene;
• feeding the input voxel-grid into a neural network to generate a multi-dimensional feature vector;
• and
• processing, using one or more fully-connected layers, the multi-dimensional feature vector to predict an excavation success probability.

A computer-implemented method for learning-based excavation planning comprising:

• generating, using an excavation planner, multiple excavation trajectories given a voxel-grid representation of an excavation scene to initialize a Gaussian distribution of trajectory parameters, each excavation trajectory is presented by multiple trajectory parameters;
• updating the Gaussian distribution through multiple iterations of excavation trajectory sampling and sampled excavation trajectory selection in terms of a predicted excavation success probability for each sampled excavation trajectory predicted by an excavation prediction model;
• sampling, from the Gaussian distribution at the last iteration, a subset of excavation trajectories;
• and
• choosing, from subset of excavation trajectories, one excavation trajectory with the highest predicted success probability as a planned excavation trajectory.