System and method for generating a classifier for semantically segmenting an image

A system for generating a classifier configured to label segments of an image, wherein the classifier comprises a first sub-classifier and a second sub-classifier, the system comprising:

• at least one processor;
• and
• a plurality of modules, each module including at least one of:
• a hardware component, or a plurality of instructions executable by the at least one processor, the modules including:
• a training module configured to train a first sub-classifier based on photographic data for a set of pre-labeled image segments and a second sub-classifier based on 3-dimensional (3D) point data for the set of pre-labeled image segments, wherein each of the pre-labeled image segments can be a portion of a larger image partitioned into a plurality of the pre-labeled image segments based on similarity of pixels within the segment and differences with pixels which are outside a boundary of the segment;
• a labeling module configured to generate a labeling solution comprising a plurality of associations between each of a plurality of image segments from a set of unlabeled image segments and a corresponding label of a plurality of labels, wherein the labeling solution is generated by running the first sub-classifier on the set of unlabeled image segments and running the second sub-classifier on the set of unlabeled image segments;
• and
• an update module configured to update the set of pre-labeled image segments based on the labeling solution generated by the first sub-classifier and the second sub-classifier wherein the update includes adding an image segment of the unlabeled image segments to the pre-labeled image segments together with a label having at least one of the associations with the added segment
• wherein the training module is further configured to retrain the first sub-classifier and the second sub-classifier based on the updated set of pre-labeled image segments.

A non-transitory machine-readable medium comprising instructions stored therein, which when executed by a machine, cause the machine to perform operations comprising:

• training a first sub-classifier based on photographic data for a set of pre-labeled image segments and a second sub-classifier based on 3-dimensional (3D) point data for the set of pre-labeled image segments, wherein each of the pre-labeled image segments can be a portion of a larger image partitioned into a plurality of the segments based on similarity of pixels within the segment and differences which are outside a boundary of the segment;
• and
• for at least one iteration:
• generating a labeling solution by running the first sub-classifier on a set of unlabeled image segments and running the second sub-classifier on the set of unlabeled image segments,
• wherein the running of the first sub-classifier on the set of unlabeled image segments is based on photographic data for the set of unlabeled image segments and wherein the running of the second sub-classifier on the set of unlabeled image segments is based on 3D point data for the set of unlabeled image segments,
• and wherein the labeling solution comprises a plurality of associations, each association of the plurality of associations linking an image segment from the set of unlabeled image segments with a label, and wherein each association is associated with a confidence value,
• updating the set of pre-labeled image segments based on the labeling solution generated by the first sub-classifier and the second sub-classifier, including adding an image segment of the unlabeled image segments to the pre-labeled image segments together with a label having at least one of the associations with the added segment, and
• training the first sub-classifier and the second sub-classifier based on the updated set of pre-labeled image segments.