Systems and methods for performing uncertainty determination in a 2d-to-3d image prediction system

A system comprising:

• a camera configured to produce a two-dimensional (2D) image of a first device;
• a processor;
• and
• non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:
• receiving the 2D image;
• predicting 2D keypoints of a target object within the 2D image based on a previously trained ensemble of neural networks;
• estimating 6 degree-of-freedom (6DOF) position (pose) of the target object using the 2D keypoints using a perspective-n-point (PnP) optimization technique to create 6DOF pose parameters for each neural network in the ensemble;
• combining the 6DOF pose parameters for each neural network in the ensemble into a single estimate of 6DOF pose parameters;
• determining an uncertainty score based on an uncertainty value derived from the single estimate of 6DOF pose parameters and an uncertainty value derived from Monte Carlo sampling of the single estimate of 6DOF pose parameters;
• and
• outputting the 2D image in response to the uncertainty score being greater than a predefined threshold.

A tanker aircraft comprising:

• a refueling boom;
• a camera configured to generate a two-dimensional (2D) image of the refueling boom;
• a processor;
• and
• non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:
• receiving the 2D image;
• predicting 2D keypoints of a target object within the 2D image based on a previously trained ensemble of neural networks;
• estimating 6 degree-of-freedom (6DOF) position (pose) of the target object using the 2D keypoints using a perspective-n-point (PnP) optimization technique to create 6DOF pose parameters for each neural network in the trained ensemble;
• combining the 6DOF pose parameters for each neural network in the ensemble into a single estimate of 6DOF pose parameters;
• determining an uncertainty score based on an uncertainty value derived from the single estimate of 6DOF pose parameters and an uncertainty value derived from Monte Carlo sampling of the single estimate of 6DOF pose parameters;
• and
• outputting the 6DOF pose parameters in response to the uncertainty score being within a predefined threshold.