Transform matrix learning for multi-sensor image capture devices

A device, comprising:

• one or more processors;
• and
• a memory coupled to the one or more processors and storing instruction that, when executed by the one or more processors, cause the device to perform operations comprising:
• receiving a first disparity value indicating a difference in field of view between a first image sensor and a second image sensor;
• receiving a first depth value corresponding to the first disparity value;
• determining a model for a disparity between the first image sensor and the second image sensor at a plurality of depth values, the model based, at least, in part, on the first disparity value and the first depth;
• receiving an input image from one of the first image sensor or the second image sensor;
• receiving a second depth value corresponding to the input image;
• determining a predicted disparity value based on the second depth value and the model;
• determining a transformation matrix for warping the input image to a field of view of the other of the first image sensor or the second image sensor;
• determining a confidence level associated with the transformation matrix is below a threshold level;
• and
• determining a corrected image based on the input image and the predicted disparity value.

A non-transitory computer-readable medium storing instructions that, when executed by a processor of a device, cause the device to perform operations comprising:

• receiving a first disparity value indicating a difference in field of view between a first image sensor and a second image sensor;
• receiving a first depth value corresponding to the first disparity value;
• determining a model for a disparity between the first image sensor and the second image sensor at a plurality of depth values, the model based, at least in part, on the first disparity value and the first depth;
• receiving an input image from one of the first image sensor or the second image sensor;
• receiving a second depth value corresponding to the input image;
• determining a predicted disparity value based on the second depth value and the model;
• determining a transformation matrix for warping the input image to a field of view of the other of the first image sensor or the second image sensor;
• determining a confidence level associated with the transformation matrix is below a threshold level;
• and
• determining a corrected image based on the input image and the predicted disparity value.

A device, comprising:

• a first image sensor configured with a first field of view;
• a second image sensor configured with a second field of view at least partially overlapping the first field of view;
• one or more processors coupled to the first image sensor and coupled to the second image sensor;
• and
• a memory coupled to the one or more processors,
• wherein the one or more processors are configured to perform steps comprising:
• receiving a first disparity value indicating a difference in field of view between the first image sensor and the second image sensor;
• receiving a first depth value corresponding to the first disparity value;
• determining a model for a disparity between the first image sensor and the second image sensor at a plurality of depth values, the model based, at least in part, on the first, disparity value and the first depth;
• receiving an input image from one of the first image sensor or the second image sensor;
• receiving a second depth value corresponding to the input image;
• determining a predicted disparity value based on the second depth value and the model;
• determining a transformation matrix for warping the input image to a field of view of the other of the first image sensor or the second image sensor;
• determining a confidence level associated with the transformation matrix is below a threshold level;
• and
• determining a corrected image based on the input image and the predicted disparity value.