System and method for tracking keypoints of an object

A method for tracking a plurality of keypoints of an object, comprising iteratively performing steps of the method at a first frame rate, wherein the steps comprise:

• receiving, via a controller comprising a processor and a memory, a ground-truth signal from a ground-truth sensor assembly indicative of a plurality of poses of the plurality of keypoints of the object at a current ground-truth frame;
• determining, via the controller, the plurality of poses of the plurality of keypoints of the object at the current ground-truth frame;
• receiving, via the controller, a set of images of the object from an optical sensor at a second frame rate, wherein the second frame rate is greater than the first frame rate;
• associating, via the controller, a plurality of points within a first image of the set of images with the plurality of keypoints of the object at a previous ground-truth frame via a calibration process;
• determining, via the controller, the plurality of poses of the plurality of keypoints of the object at each frame of one or more intermediate frames based on the set of images via an optical flow process;
• and
• outputting, via the controller at the second frame rate, a plurality of pose signals based on the plurality of poses of the plurality of keypoints of the object at a plurality of frames, wherein the plurality of frames comprises the previous ground-truth frame and the one or more intermediate frames.

A tracking system for a plurality of keypoints of an object, comprising:

• a ground-truth sensor assembly configured to output a ground-truth signal indicative of a plurality of poses of the plurality of keypoints of the object;
• an optical sensor configured to output images of the object;
• and
• a controller communicatively coupled to the ground-truth sensor assembly and to the optical sensor, wherein the controller comprises a memory and a processor, and the controller is configured to iteratively perform a keypoint tracking process at a first frame rate, and the keypoint tracking process comprises:
• receiving the ground-truth signal from the ground-truth sensor assembly indicative of the plurality of poses of the plurality of keypoints of the object at a current ground-truth frame;
• determining the plurality of poses of the plurality of keypoints of the object at the current ground-truth frame;
• receiving a set of the images of the object from the optical sensor at a second frame rate, wherein the second frame rate is greater than the first frame rate;
• associating a plurality of points within a first image of the set of images with the plurality of keypoints of the object at a previous ground-truth frame via a calibration process;
• determining the plurality of poses of the plurality of keypoints of the object at each frame of one or more intermediate frames based on the set of images via an optical flow process;
• and
• outputting, at the second frame rate, a plurality of pose signals based on the plurality of poses of the plurality of keypoints of the object at a plurality of frames, wherein the plurality of frames comprises the previous ground-truth frame and the one or more intermediate frames.