Dual camera hmd with remote camera alignment

A method for aligning and stabilizing images generated by an integrated stereo camera pair comprising a first camera and a second camera that are physically mounted to a computer system with images generated by a detached camera that is physically unmounted from the computer system, said method comprising:

• generating a first image using the first camera, generating a second image using the second camera, and generating a third image using the detached camera;
• computing a first rotation base matrix of the third image relative to the first image;
• computing a second rotation base matrix of the third image relative to the second image;
• aligning the third image to the first image using the first rotation base matrix and aligning the third image to the second image using the second rotation base matrix;
• generating a first overlaid image by overlaying at least a first portion of the third image onto the first image based on said aligning;
• generating a second overlaid image by overlaying at least a second portion of the third image onto the second image based on said aligning;
• performing a parallax correction on the first overlaid image by modifying the first overlaid image from a first perspective to a first new perspective;
• and
• displaying the first overlaid image and the second overlaid image,
• wherein performing the parallax correction on the first overlaid image is performed by:
• identifying pixels included within the third image;
• selecting a depth corresponding to at least one of the identified pixels;
• attributing the depth to all of the identified pixels;
• and
• when performing the first parallax correction, reprojecting the identified pixels using the attributed depth.

A computer system configured to align and stabilize images generated by an integrated stereo camera pair comprising a first camera and a second camera that are physically mounted to the computer system with images generated by a detached camera that is physically unmounted from the computer system, said computer system comprising:

• one or more processors;
• and
• one or more computer-readable tangible hardware storage devices that store instructions that are executable by the one or more processors to cause the computer system to at least:
• generate a first image using the first camera, generating a second image using the second camera, and generating a third image using the detached camera;
• compute a first rotation base matrix of the third image relative to the first image;
• compute a second rotation base matrix of the third image relative to the second image;
• align the third image to the first image using the first rotation base matrix and aligning the third image to the second image using the second rotation base matrix;
• generate a first overlaid image by overlaying the third image onto the first image based on said aligning;
• generate a second overlaid image by overlaying the third image onto the second image based on said aligning;
• perform a first parallax correction on the first overlaid image by modifying the first overlaid image from a first perspective to a first new perspective;
• perform a second parallax correction on the second overlaid image by modifying the second overlaid image from a second perspective to a second new perspective;
• and
• display the first overlaid image and the second overlaid image,
• wherein performing the first parallax correction on the first overlaid image is performed by:
• identifying pixels included within the third image;
• selecting a depth corresponding to at least one of the identified pixels;
• attributing the depth to all of the identified pixels;
• and
• when performing the first parallax correction, reprojecting the identified pixels using the attributed depth.