Flight safety demonstration and infotainment through mixed reality
AssigneeRockwell Collins, Inc.
InventorsPoornima Kathiresan, Aditya Kumar, Santhosh Ramamurthy
Life of this patent
Estimating the term…
Abstract
The present disclosure is generally directed to the use of mixed reality to provide safety information as well as infotainment during flight. Demonstrations of the safety information for the aircraft are augmented in a 3-D way with a virtual representation of the aircraft. The virtual representation provides the safety instructions to the passengers. The mixed reality concept provides the safety information with the help of video-see through mixed reality by way of a head-mounted display worn by the passengers. The safety instructions are provided with a complete working view of locations and operations for the safety jackets, emergency exits, and oxygen passenger service units.
Claims15
A computer-implemented method comprising:
- receiving a first video signal and pose information;
- wherein the first video signal is received from a camera of a head-mounted display within an aircraft environment, the first video signal corresponding to images captured of the aircraft environment;
- wherein the pose information is received from a head tracker of the head-mounted display;
- wherein the first video signal and the pose information are received by way of an avionics network;
- detecting a feature vector within the first video signal by digital image processing the first video signal using a feature detector algorithm, the feature vector including at least a first feature, the first feature corresponding to an aircraft safety device, the aircraft safety device including one of a life preserver stowage, an oxygen mask passenger service unit, a seatbelt, or an emergency exit door;
- estimating a pose of the aircraft safety device within the first video signal based on the feature vector together with the pose information from the head tracker;
- generating a second video signal of the aircraft environment, the second video signal including the first video signal and one or more virtual objects composited with the first video signal, the one or more virtual objects corresponding to a safety instruction associated with the aircraft safety device, the one or more virtual objects positioned and oriented within the second video signal from a perspective of the head-mounted display based on the pose of the aircraft safety device;
- and
- causing a display of the head-mounted display to display the second video signal.
An aircraft system comprising:
- a head-mounted display within an aircraft environment, the head-mounted display including;
- a frame;
- at least one camera fixed to the frame and configured to generate a first video signal corresponding to images captured of the aircraft environment;
- a head tracker fixed to the frame and configured to generate pose information indicative of a position and orientation of the at least one camera;
- a display fixed to the frame and configured to generate an image;
- an avionics network;
- and
- a processor communicatively coupled to the head-mounted display by the avionics network, the processor configured to execute program instructions maintained on a memory causing the processor to:
- receive the first video signal and the pose information from the head-mounted display by way of the avionics network;
- detect a feature vector within the first video signal by digital image processing the first video signal using a feature detector algorithm, the feature vector including at least a first feature corresponding to an aircraft safety device including one of a life preserver stowage, an oxygen mask passenger service unit, a seatbelt, or an emergency exit door;
- estimate a pose of the aircraft safety device within the first video signal based on the feature vector together with the pose information from the head tracker;
- generate a second video signal by compositing the first video signal with one or more virtual objects corresponding to a safety instruction associated with the aircraft safety device, the one or more virtual objects positioned and oriented within the second video signal from a perspective of the head-mounted display based on the pose of the aircraft safety device;
- and
- cause the display of the head-mounted display to display the second video signal.
Dependencies are inferred from the claim text.