Automated semantic segmentation of non-euclidean 3d data sets using deep learning

A computer-implemented method for training a deep neural network DNN to semantically segment a point cloud, the method comprising:

• a computer receiving training data, the training data comprising one or more point clouds, each of the one or more point clouds representing a predetermined object, and each point of the one or more point clouds being associated with a target label;
• determining subsets of the one or more point clouds of the training data using a non-uniform resampling algorithm, each of the one or more subsets being associated with target labels and each of the one or more subsets including a first number of points arranged within a predetermined spatial distance of a selected point of the point cloud and a second number of points arranged at spatial distances larger than the predetermined spatial distance, the first number of point representing one or more fine features of the object around the selected point and the second number of points representing one or more global features of the object;
• providing one of the one or more subset to the input of the DNN and receiving from the output of the DNN and for each point of the subset that is provided to the input of the DNN, receiving at the output of the DNN a multi-element vector, wherein each element of the vector represents a probability that the point belongs to one of the plurality of classes of the object;
• and, training the parameters of the DNN by minimizing a loss function representing a deviation between labels predicted by the DNN and the target labels.

A computer system, preferably a server system, adapted to semantically segment a point cloud comprising:

• a computer readable storage medium having computer readable program code embodied therewith, the program code including a non-uniform sampling module and at least a trained deep neural network DNN, the computer readable program code;
• and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the first computer readable program code, the processor is configured to perform executable operations comprising:
• receiving a point cloud, the point cloud including points represented by a vector in a 3D space, the points representing a predetermined object;
• determining one or more subsets of the point cloud using a non-uniform resampling algorithm, each of the one or more subsets including a first number of points arranged within a predetermined spatial distance of a selected point of the point cloud and a second number of points arranged at spatial distances larger than the predetermined spatial distance, the first number of point representing one or more fine features of the object around the selected point and the second number of points representing one or more global features of the object;
• providing each of one or more subsets of points to the input of a deep neural network, DNN, the deep neural network being trained to semantically segment points of each of the one or more subsets that is provided to the input of the DNN according to a plurality of classes associated with the object;
• and,
• for each point of the subset that is provided to the input of the DNN, receiving at the output of the DNN a multi-element vector, wherein each element of the vector represents a probability that the point belongs to one of the plurality of classes of the object.