Final view generation using offset and/or angled see-through cameras in video see-through (VST) extended reality (XR)

A video see-through (VST) extended reality (XR) device comprising:

• see-through cameras configured to capture images of a scene, wherein the see-through cameras are at least one of (i) offset from forward axes extending from expected locations of a user's eyes when using the VST XR device or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes;
• one or more display panels;
• and
• at least one processing device configured to:
• identify a passthrough transformation associated with the VST XR device;
• apply the passthrough transformation to the images in order to generate transformed images;
• and
• initiate display of the transformed images on the one or more display panels;
• wherein the passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.

A non-transitory machine readable medium containing instructions that when executed cause at least one processor of a video see-through (VST) extended reality (XR) device to:

• identify a passthrough transformation associated with the VST XR device, the VST XR device comprising see-through cameras that are at least one of (i) offset from forward axes extending from expected locations of a user's eyes when using the VST XR device or (ii) rotated such that optical axes of the see-through cameras are angled relative to the forward axes;
• obtain images of a scene captured using the see-through cameras;
• apply the passthrough transformation to the images in order to generate transformed images;
• and
• initiate display of the transformed images on one or more display panels of the VST XR device;
• wherein the passthrough transformation is based on (i) a first transformation between see-through camera viewpoints and viewpoint-matched virtual camera viewpoints and (ii) a second transformation that aligns principal points of the see-through cameras and principal points of the one or more display panels.