Planar splatting

An apparatus to process one or more images, the apparatus comprising:

• one or more memories configured to store one or more images;
• and
• one or more processors coupled to the one or more memories and configured to:
• segment, using a first neural network, the one or more images of a scene to determine a respective plurality of segments for each image of the one or more images;
• determine, using a second neural network, a respective plurality of normal vectors for each image of the one or more images, wherein each normal vector of the respective plurality of normal vectors is orthogonal to a surface associated with a respective primitive of a plurality of primitives associated with an image of the one or more images;
• generate a graph based on each respective plurality of segments for each image, each respective plurality of normal vectors for each image, and a plurality of estimated planar distances;
• partition, based on each respective plurality of normal vectors and the plurality of estimated planar distances, the graph to determine a plurality of indexes associated with the plurality of primitives;
• assign each descriptor of a plurality of descriptors to an index of the plurality of indexes based on a respective weight;
• and
• merge primitives of the plurality of primitives with associated indexes of the plurality of indexes that are similar to each other.