System and method for multi-client deployment of augmented reality instrument tracking

A system for performing guiding augmented reality (AR) intervention for planning, intervention, guidance, and/or education for medical applications, wherein the system comprises:

• a server including:
• a database having:
• a plurality of data models that each have a plurality of model set records, a plurality of plans records, a plurality of recordings records, and a plurality of instruments records;
• a plurality of user records;
• and
• a plurality of session records;
• and
• at least one processor that is operatively coupled to the database and configured to execute program instructions for implementing:
• an HTTP server for providing endpoints for queries and delivery of content, user authentication, and management of sessions;
• and
• a WebSocket server to enable multi-client broadcast of data across device specific listening channels by setting up WebSocket clients;
• and
• a primary client device that is communicatively coupled to the server to interact with the HTTP server and the WebSocket server, the primary client device including a first processor and a first input device, the primary client device being configured to:
• receive model sets, an intervention plan having an intervention field, and session information about a session related to the AR intervention from the server;
• receive first real-time input data from the first input device;
• generate metrics by determining an evaluation of an execution of the intervention plan by comparing the intervention plan to the first real-time input data;
• display real-time graphics, based at least in part on the metrics, spatially over the intervention field;
• receive real-time status data, from the server, about a replicate client device connected to the server after the replicate client device joins the session;
• send the first real-time input data, through the server, to the replicate client device within the session;
• and
• send the metrics and the evaluation computed from the intervention plan, through the server, to the replicate client device within the session.