Transformation for covariate shift of grasp neural networks

A method of grasping an object, the method comprising:

• capturing an image of the object within a physical environment, each of the object and the physical environment defining a plurality of locations, the image defining a plurality of pixels representative of the plurality of locations of the object and the physical environment;
• sending the image of the object within the physical environment to a neural network;
• based on the image, the neural network generating an output that defines respective grasp scores associated with the plurality of pixels, the grasp scores indicating a respective grasp quality at the locations of the object represented by the respective pixel;
• transforming the output of the neural network, wherein transforming the output of the neural network comprises:
• sorting the grasp scores in order of grasp quality and, for grasp locations associated with adjacent grasp scores in the sorted list, determining respective distances, comparing the distances to a minimum distance, and rejecting a highest-ranked grasp location as an outlier based on a determination that the distance to a next highest-ranked location exceeds the minimum distance;
• and
• extracting a grasp point and surrounding points to generate a point cloud, fitting a plane through the points by least squares, computing residuals of the points to the plane, comparing the residuals to a predetermined threshold to detect whether an edge is near the grasp point, and, based on a determination that an edge is detected, moving the grasp point away from the edge by a predetermined distance based on a size or geometry of an end effector;
• based on transforming the output of the neural network, selecting a grasp location of the plurality of locations of the object;
• grasping the object, by an autonomous machine comprising an end effector, at the grasp location.