US 2025/0166342 A1 · MONOCULAR 2D SEMANTIC KEYPOINT DETECTION AND TRACKING

Abstract

A method for 2D semantic keypoint detection and tracking is described. The method includes learning embedded descriptors of salient object keypoints detected in previous images according to a descriptor embedding space model. The method also includes predicting, using a shared image encoder backbone, salient object keypoints within a current image of a video stream. The method further includes inferring an object represented by the predicted, salient object keypoints within the current image of the video stream. The method also includes tracking the inferred object by matching embedded descriptors of the predicted, salient object keypoints representing the inferred object within the previous images of the video stream based on the descriptor embedding space model.

Claims16

Dependent claims

A method for 2D semantic keypoint detection and tracking, comprising:

generating, by a trained shared descriptor embedding space model, a joint descriptor embedding space, including embedded descriptors of predicted salient object keypoints in a previous frame of a video stream and embedded descriptors of the predicted salient object keypoints within a current frame of the video stream;
matching the embedded descriptors from the joint descriptor embedding space to identify the predicted salient object keypoints within the current frame that correspond with the predicted salient object keypoints within the previous frame;
inferring an object represented by the corresponding predicted, salient object keypoints within the current frame of the video stream;
and
tracking the inferred object within the previous frames, the current frame, and subsequent frames of the video stream based on the matching.

2 more elements

A non-transitory computer-readable medium having program code recorded thereon for 2D semantic keypoint detection and tracking, the program code being executed by a processor and comprising:

program code to generate, by a trained shared descriptor embedding space model, a joint descriptor embedding space, including embedded descriptors of predicted salient object keypoints in a previous frame of a video stream and embedded descriptors of the predicted salient object keypoints within a current frame of the video stream;
program code to match the embedded descriptors from the joint descriptor embedding space to identify the predicted salient object keypoints within the current frame that correspond with the predicted salient object keypoints within the previous frame;
program code to infer an object represented by the corresponding predicted, salient object keypoints within the current frame of the video stream;
and
program code to track the inferred object within the previous frames, the current frame, and subsequent frames of the video stream based on the matching.

2 more elements

Dependencies are inferred from the claim text.

Monocular 2d semantic keypoint detection and tracking

Life of this patent

Monocular 2d semantic keypoint detection and tracking

Life of this patent

Abstract

Claims16