Motion estimation with depth information

An apparatus for aligning images, the apparatus comprising:

• at least one memory;
• and
• at least one processor coupled to the at least one memory and configured to:
• determine a plurality of feature points in a first image;
• determine, based on the plurality of feature points, a plurality of motion vectors associated with the plurality of feature points;
• determine background motion vectors of the plurality of motion vectors associated with a background of a scene of the first image;
• determine, based on the background motion vectors, a transformation matrix for aligning the background of the first image and the background of a second image;
• determine a scaling factor based on a magnitude of motion vectors of the plurality of motion vectors within a portion of a foreground of the scene of the first image;
• and
• scale, based on the scaling factor, the transformation matrix to generate a local transformation matrix for aligning the portion of the foreground of the scene of the first image with a corresponding portion of the foreground of the scene of the second image.