Network architecture for ego-motion estimation

A method for estimating ego-motion based on a plurality of input images in a self-supervised system, comprising:

• receiving a source image and a target image;
• determining a depth estimation Dt based on the target image;
• determining a depth estimation Ds based on a source image;
• determining an ego-motion estimation in a form of a six degrees-of-freedom (6 DOF) transformation between the target image and the source image by inputting the depth estimations (Dt, Ds), the target image, and the source image into a two-stream network architecture trained to output the 6 DOF transformation based at least in part on the depth estimations (Dt, Ds), the target image, and the source image,
• wherein determining the ego-motion estimation comprises:
• convolving the source image and target image via an appearance stream convolution neural network (CNN);
• convolving the depth estimations (Dt, Ds) via a structure stream CNN;
• and
• fusing outputs of the appearance stream CNN and the structure stream CNN into a unified output to produce the 6 DOF transformation.

A non-transitory computer-readable medium for estimating ego-motion based on a plurality of input images in a self-supervised system, including instructions that, when executed by one or more processors, cause the one or more processors to:

• receive a source image and a target image;
• determine a depth estimation Dt based on the target image;
• determine a depth estimation Ds based on a source image;
• determine an ego-motion estimation in a form of a six degrees-of-freedom (6 DOF) transformation between the target image and the source image by inputting the depth estimations (Dt, Ds), the target image, and the source image into a two-stream network architecture trained to output the 6 DOF transformation based at least in part on the depth estimations (Dt, Ds), the target image, and the source image;
• and
• determine the ego-motion estimation by:
• convolving the source image and target image via an appearance stream convolution neural network (CNN);
• convolving the depth estimations (Dt, Ds) via a structure stream CNN;
• and
• fusing outputs of the appearance stream CNN and the structure stream CNN into a unified output to produce the 6 DOF transformation.