Cross spectral feature mapping for camera calibration

A three-dimensional (3D) teleconferencing system comprising:

• a memory including code segments representing a plurality of computer instructions;
• and
• a processor configured to execute the code segments, the computer instructions including:
• capturing a first image of a real-world scene by a first camera sensitive to a first spectrum of light, the first camera having a first light source;
• capturing a second image of the real-world scene by a second camera sensitive to a second spectrum of light, the second camera having a second light source;
• identifying at least one feature in the first image;
• identifying, using a machine learning (ML) model, at least one feature in the second image that matches the at least one feature identified in the first image;
• mapping pixels in the first image and the second image to rays in a three-dimensional (3D) space based on the matched at least one feature;
• and
• calibrating the first camera and the second camera based on the mapping.

A non-transitory computer readable medium containing instructions that when executed cause a processor of a computer system to perform steps comprising:

• capturing a first image of a real-world scene by a first camera sensitive to a first spectrum of light, the first camera having a first light source;
• capturing a second image of the real-world scene by a second camera sensitive to a second spectrum of light, the second camera having a second light source;
• identifying at least one feature in the first image;
• identifying, using a machine learning (ML) model, at least one feature in the second image that matches the at least one feature identified in the first image;
• mapping pixels in the first image and the second image to rays in a three-dimensional (3D) space based on the matched at least one feature;
• and
• calibrating the first camera and the second camera based on the mapping.