Semantic point cloud map localization and mapping

A point cloud based semantic segmentation system comprising:

• a first vehicle configured to traverse an external environment, the first vehicle comprising:
• at least a first imaging sensor configured to capture a first point cloud representation of the external environment;
• at least a first vehicle position sensor configured to measure first odometry information related to an orientation, a velocity, and an acceleration of the first vehicle;
• a first Electronic Control Unit (ECU) comprising:
• a first processor configured to:
• receive the first captured point cloud representation of the external environment from at least the first imaging sensor;
• associate a location of the first captured point cloud representation of the external environment with a location of the first vehicle on Earth based on the first odometry information;
• a first memory configured to store the first captured point cloud representation of the external environment;
• a first transceiver configured to transmit the first captured point cloud representation;
• a server configured to generate a map of the external environment, the server comprising:
• a second transceiver configured to receive the first captured point cloud representation of the external environment from the first vehicle;
• a second memory configured to store a mapping module comprising computer readable code;
• a second processor configured to execute the computer readable code forming the mapping module, where the computer readable code causes the second processor to:
• perform semantic segmentation on a plurality of features in the first captured point cloud representation;
• generate the map of the external environment including the semantically segmented features of the first captured point cloud representation;
• a second vehicle configured to localize the second vehicle on the generated map of the external environment, the second vehicle comprising:
• at least a second imaging sensor configured to capture a second point cloud representation of the external environment;
• at least a second vehicle position sensor configured to measure second odometry information related to an orientation, a velocity, and an acceleration of the second vehicle;
• a second ECU comprising:
• a third transceiver configured to receive the generated map of the external environment from the server;
• a third memory configured to store the second captured point cloud representation of the external environment and the generated map;
• a third processor configured to:
• receive the second captured point cloud representation of the external environment from at least the second imaging sensor;
• localize the second vehicle on the generated map of the external environment based on the second captured point cloud representation of the external environment and the second odometry information.