Estimation of spatial relationships between sensors of a multi-sensor device

A device comprising:

• a first visible spectrum image sensor;
• a second visible spectrum image sensor;
• a depth sensor;
• a non-transitory computer-readable storage medium;
• and
• one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the system to perform operations comprising:
• capturing, via the first visible spectrum image sensor, a first image comprising first image features;
• estimating 3D positions for the first image features based on depth information from the depth sensor and a first extrinsic parameter corresponding to a spatial relationship between the depth sensor and the first visible spectrum image sensor;
• capturing, via the second visible spectrum image sensor, a second image comprising second image features;
• matching the first image features and the second image features;
• and
• based on the matching and estimated 3D positions for the first image features, estimating a second extrinsic parameter corresponding to a spatial relationship between the first visible spectrum image sensor and the second visible spectrum image sensor.

A non-transitory computer-readable storage medium storing program instructions executable via one or more processors to perform operations comprising:

• capturing, via the first visible spectrum image sensor, a first image comprising first image features;
• estimating 3D positions for the first image features based on depth information from the depth sensor and a first extrinsic parameter corresponding to a spatial relationship between the depth sensor and the first visible spectrum image sensor;
• capturing, via the second visible spectrum image sensor, a second image comprising second image features;
• matching the first image features and the second image features;
• and
• based on the matching and estimated 3D positions for the first image features, estimating a second extrinsic parameter corresponding to a spatial relationship between the first visible spectrum image sensor and the second visible spectrum image sensor.