System and method for superimposing spatially correlated data over live real-world images

A mobile computing device, comprising:

• a display screen;
• an optical sensor that acquires a live real-time image to be displayed by the display screen;
• a non-optical sensor;
• at least one computer processor;
• and
• a non-transitory computer readable medium storing instructions that, when executed by the at least one computer processor, cause the mobile computing device to:
• read modeled facility data stored by a remote database device that is external to the mobile computing device and forms part of a computerized facility system, over a wireless communication network, into a memory of the mobile computing device, wherein the modeled facility data includes a virtual element representing a hidden element of the facility and a virtual element representing an unhidden element of the facility that are spatially correlated to positions in a three-dimensional space;
• generate mobile position data based on non-optical sensor readings taken by the non-optical sensor of the mobile computing device, wherein the mobile position data represents at least a location and an orientation of the mobile computing device in a three-dimensional space of a facility;
• generate projection parameters based at least in part on mobile position data stored in a database, wherein the projection parameters are data representing a spatial arrangement of the virtual element representing the unhidden element of the facility represented by modeled facility data, relative to a spatial arrangement of a live image of the unhidden element of the facility appearing in the live real-time image acquired by the optical sensor, according to a view of the optical sensor, for projecting and spatially aligning the virtual element representing the unhidden element and the virtual element representing the hidden element from the modeled facility data onto locations corresponding to the live image of the unhidden element and a position of the hidden element in the live real-time image,
• generate a rendered image at least in part by applying the projection parameters to the modeled facility data representing the virtual element representing the unhidden element and the virtual element representing the hidden element of the facility within the three-dimensional space, and
• superimpose the rendered image onto the live real-time image acquired by the optical sensor of the mobile computing device, wherein the live real-time image includes the live image of the unhidden element of the facility within the three-dimensional space, and wherein the virtual element representing the hidden element and the virtual element representing the unhidden element of the modeled facility in the rendered image are spatially aligned with respective locations of the position of the hidden element and the live image of the unhidden element in the live real-time image within the three-dimensional space.

A non-transitory computer-readable medium storing computer-executable instructions that, when executed by at least one computer processor of a mobile computing device, cause the mobile computing device to:

• read modeled facility data stored by a remote database device that is external to the mobile computing device and forms part of a computerized facility system, over a wireless communication network, and store the modeled facility data into a memory of the mobile computing device, wherein the modeled facility data includes a virtual element representing a hidden element of the facility and a virtual element representing an unhidden element of the facility that are spatially correlated to positions in a three-dimensional space;
• generate mobile position data based on non-optical sensor readings taken by the mobile computing device, wherein the mobile position data represents at least a location and an orientation of the mobile computing device in a three-dimensional space of a facility;
• generate projection parameters based at least in part on the mobile position data, wherein the projection parameters are data representing a spatial arrangement of the virtual element representing the an unhidden element of the facility represented by the modeled facility data read from the remote database, relative to a spatial arrangement of a live image of the unhidden element of the facility appearing in a live real-time image acquired by an optical sensor of the mobile computing device, according to a view of the optical sensor, for projecting and spatially aligning a virtual element representing the hidden element and the virtual element representing the unhidden element from the modeled facility data of the facility onto locations corresponding to the unhidden element and the hidden element in the live real-time image;
• generate a rendered image at least in part by applying the projection parameters to the modeled facility data representing the virtual element representing the unhidden element and the virtual element representing the hidden element of the facility within the three-dimensional space;
• and
• superimpose the rendered image onto the live real-time image acquired by the optical sensor of the mobile computing device, wherein the live real-time image includes the live image of the unhidden element of the facility within the three-dimensional space, and wherein the virtual element representing the hidden element and the virtual element representing the unhidden element of the modeled facility in the rendered image are spatially aligned with respective locations of the position of the hidden element and the live image of the unhidden element in the live real-time image within the three-dimensional space.