Overlay technology for enhancing connectivity and realism in interacting simulations

Method providing interactive simulation comprising;

• a first computer displaying a first simulation interacting with a first user, the first computer is connected to a first head mounted display (HMD), and a first interactive control;
• a second computer displaying a second simulation interacting with a second user, the second computer is connected to a second HMD and a second interactive control, wherein the first computer is connected to the second computer via a network, and wherein both the first and second simulations are not designed to interact by providing the behaviours or even the movements of the different users characterized by:
• a first overlay program running on the first computer and second overlay program running on the second computer, wherein the first overly program utilizes a first overlay engine that writes graphic information into a graphic card memory of the first computer displayed in the first HMD, to at least partially overlay graphic information of the first simulation, and the overly program utilizes a second overlay engine that writes graphic information into a graphic card memory of the second computer displayed in the second HMD to at least partially overlay graphic information of the second simulation without any other connection of the second overlay to the second simulation;
• a first network-enabled communication module that facilitates the exchange of simulation state information, user control inputs, and overlay rendering data between simulation software and overlay systems via standardized interfaces and protocols running on the first computer and a second such module running on the second computer, the first network-enabled communication module communicating with the first simulation, via application programming interfaces (APIs) or standard interfaces, for receiving and/or setting status information of the first simulation, the first network-enabled communication module further communicating with the first overlay program to provide first overlay information;
• a second networked-enabled communication module communicating with the second simulation, via APIs or standard interfaces, for receiving and/or setting status information of the second simulation, the second network-enabled communication module further communicating with the second overlay program to provide second overlay information;
• wherein the first and second network-enabled communication modules communicate with each other over the network to exchange status information and to generate graphic information representing the other user in the simulations, displaying the respective user and their operation as an overlay over the simulation on the corresponding HMD;
• and
• wherein a number of computers, simulations, and HMDs is at least two, all of which communicating with each other and exchanging overlay information and simulation status information, to generate a common interactive simulation experience.