Aerial refueling systems and methods

A tanker aircraft comprising:

• a refueling boom;
• a camera configured to generate a two-dimensional (2D) image of an in-flight refueling operation between a receiver aircraft and the tanker aircraft;
• a processor;
• and
• non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:
• cropping the 2D image based on predefined feature areas of interest of the receiver aircraft to produce a plurality of cropped images;
• resizing one or more of the plurality of cropped images responsive to the one or more of the plurality of cropped images having a resolution different from a training resolution associated with training of a previously trained keypoint detection neural network to produce one or more resized images, the one or more resized images having a resized resolution equal to the training resolution;
• determining 2D keypoints of the receiver aircraft within a plurality of input images by inputting the plurality of input images into the keypoint detection neural network, the plurality of input images comprising the one or more resized images;
• estimating a 6 degrees-of-freedom (6DOF) pose based on the 2D keypoints and a three-dimensional (3D) model of the receiver aircraft to produce an estimated 6DOF pose;
• and
• outputting the 6DOF pose;
• wherein the plurality of cropped images comprises a first cropped image and a second cropped image and neither the first cropped image nor the second cropped image is entirely contained within the other.

A system comprising:

• a camera configured to generate a two-dimensional (2D) image of an in-flight refueling operation between a receiver aircraft and a tanker aircraft;
• a processor;
• and
• non-transitory computer readable storage media storing code, the code being executable by the processor to perform operations comprising:
• cropping the 2D image based on predefined feature areas of interest of the receiver aircraft to produce a plurality of cropped images;
• resizing one or more of the plurality of cropped images responsive to the one or more of the plurality of cropped images having a resolution different from a training resolution associated with training of a previously trained keypoint detection neural network to produce one or more resized images, the one or more resized images having a resized resolution equal to the training resolution;
• determining 2D keypoints of the receiver aircraft within a plurality of input images by inputting the plurality of input images into the keypoint detection neural network, the plurality of input images comprising the one or more resized images;
• estimating a 6 degrees-of-freedom (6DOF) pose based on the 2D keypoints and a three-dimensional (3D) model of the receiver aircraft to produce an estimated 6DOF pose;
• and
• outputting the 6DOF pose;
• wherein the plurality of cropped images comprises a first cropped image and a second cropped image and neither the first cropped image nor the second cropped image is entirely contained within the other.