Learning reliable keypoints in situ with introspective self-supervision

An apparatus comprising:

• one or more processors to:
• provide a view-overlapped keyframe pair from a pose graph that is generated by a visual simultaneous localization and mapping (VSLAM) process executed by the one or more processors;
• determine a keypoint match from the view-overlapped keyframe pair based on a keypoint detection and matching process, the keypoint match corresponding to a keypoint;
• calculate an inverse reliability score based on matched pixels corresponding to the keypoint match in the view-overlapped keyframe pair;
• identify a supervision signal associated with the keypoint match, the supervision signal comprising a keypoint reliability score of the keypoint based on a final pose output of the VSLAM process;
• and
• train a keypoint detection neural network using the keypoint match, the inverse reliability score, and the keypoint reliability score.

A non-transitory computer-readable storage medium having stored thereon executable computer program instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:

• providing a view-overlapped keyframe pair from a pose graph that is generated by a visual simultaneous localization and mapping (VSLAM) process executed by the one or more processors;
• determining a keypoint match from the view-overlapped keyframe pair based on a keypoint detection and matching process, the keypoint match corresponding to a keypoint;
• calculating an inverse reliability score based on matched pixels corresponding to the keypoint match in the view-overlapped keyframe pair;
• identifying a supervision signal associated with the keypoint match, the supervision signal comprising a keypoint reliability score of the keypoint based on a final pose output of the VSLAM process;
• and
• training a keypoint detection neural network using the keypoint match, the inverse reliability score, and the keypoint reliability score.

A method for facilitating learning reliable keypoints in situ with introspective self-supervision, the method comprising:

• providing a view-overlapped keyframe pair from a pose graph that is generated by a visual simultaneous localization and mapping (VSLAM) process executed by the one or more processors;
• determining a keypoint match from the view-overlapped keyframe pair based on a keypoint detection and matching process, the keypoint match corresponding to a keypoint;
• calculating an inverse reliability score based on matched pixels corresponding to the keypoint match in the view-overlapped keyframe pair;
• identifying a supervision signal associated with the keypoint match, the supervision signal comprising a keypoint reliability score of the keypoint based on a final pose output of the VSLAM process;
• and
• training a keypoint detection neural network using the keypoint match, the inverse reliability score, and the keypoint reliability score.