Video sensor fusion and model based virtual and augmented reality systems and methods

An apparatus comprising:

• a logic device configured to communicate with a plurality of navigational sensors, at least one orientation and/or position sensor (OPS), and an imaging module coupled to a mobile structure, wherein each navigational sensor is configured to provide navigational data associated with the mobile structure, and wherein the logic device is configured to:
• receive at least one image from the imaging module;
• receive orientation, position, and/or navigation data corresponding to the at least one image from the OPS and/or the plurality of navigational sensors;
• generate an augmented reality (AR) display view for viewing by a user of the mobile structure, the AR display view comprising the at least one image and an AR graphics overlay based, at least in part, on the received orientation, position, and/or navigation data and/or the received at least one image;
• determine a boundary of a cross track error corresponding to a track traversed by the mobile structure;
• and
• render, in the AR display view, a cross track boundary line configured to identify the boundary of the cross track error within the at least one image and a traversal direction associated with the track.