System and method for three-dimensional augmented reality guidance for use of equipment

A guidance system for providing real-time, three-dimensional (3D) augmented reality (AR) feedback guidance in the use of an equipment system, the guidance system comprising:

• a computer comprising:
• an equipment interface to an equipment system, wherein said equipment interface is adapted to receive data from the equipment system during an equipment procedure performed by a user of the equipment to achieve an equipment procedure outcome;
• an AR user interface (ARUI) to an AR head mounted display (HMD) for presenting information pertaining to both real and virtual objects to the user during the performance of the equipment procedure;
• a guidance system interface (GSI) to a three-dimensional guidance system (3DGS) that senses real-time, three-dimensional user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system within a volume of a user's environment during the equipment procedure performed by the user;
• a library containing 1) stored reference positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system during a reference performance of the equipment procedure and 2) stored reference outcome data relating to an outcome of a reference performance of the equipment procedure;
• and
• a machine learning module (MLM) for providing at least one of 1) real-time position-based 3D AR feedback to the user based on the sensed user positioning data, and 2) real-time outcome-based 3D AR feedback to the user based on the equipment procedure outcome, the MLM comprising:
• a position-based feedback module comprising:
• a first module for receiving and analyzing the real-time, three-dimensional user positioning data;
• a second module for comparing the real-time, three-dimensional user positioning data to the stored reference positioning data and generating an output;
• and
• a third module for generating real-time position-based 3D AR feedback based on the output of the second module, and providing said real-time position-based 3D AR feedback to the user via the ARUI;
• and
• an outcome-based feedback module comprising:
• a fourth module for receiving real-time data from the equipment system via said equipment interface as the user performs the equipment procedure, wherein the real-time data is related to an outcome of the equipment procedure;
• a fifth module for comparing the real-time data related to an outcome of the equipment procedure received from the fourth module to the stored reference outcome data, and for generating an output;
• and
• a sixth module for generating real-time outcome-based 3D AR feedback based on the output of the fifth module, and providing said real-time outcome-based 3D AR feedback to the user via the ARUI.

A guidance system for providing real-time, three-dimensional (3D) augmented reality (AR) feedback guidance to a user of an equipment system, the guidance system comprising:

• an equipment interface to an equipment system, wherein said equipment interface is capable of receiving data from the equipment system during an equipment procedure performed by a user of the equipment system;
• an augmented reality user interface (ARUI) for presenting data pertaining to both real and virtual objects to the user during at least a portion of the performance of the equipment procedure by the user;
• a three-dimensional guidance system (3DGS) that is capable of sensing real-time, three-dimensional user positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system during said equipment procedure performed by the user;
• a library containing 1) stored reference positioning data relating to one or more of the movement, position, and orientation of at least a portion of the equipment system during a reference performance of the equipment procedure and 2) stored reference outcome data relating to an outcome of said reference equipment procedure;
• and
• a machine learning module (MLM) for providing 1) real-time position-based 3D AR feedback to the user based on the sensed user positioning data and the reference positioning data, and 2) real-time outcome-based 3D AR feedback to the user based on data received from the equipment system during the equipment procedure performed by the user and the reference outcome data, the MLM comprising:
• a position-based feedback module comprising:
• a first module for receiving and analyzing the real-time, three-dimensional user positioning data sensed by the 3DGS during the equipment procedure performed by the user;
• a second module for comparing the real-time, three-dimensional user positioning data to the stored reference positioning data and generating an output;
• and
• a third module for generating real-time position-based 3D AR feedback based on the output of the second module, and providing said real-time position-based 3D AR feedback to the user via the ARUI;
• and
• an outcome-based feedback module comprising:
• a fourth module for receiving real-time data from the equipment system via said equipment interface during the equipment procedure performed by the user, wherein the real-time data is related to an outcome of the equipment procedure;
• a fifth module for comparing the real-time data related to an outcome of the equipment procedure received from the fourth module to the stored reference outcome data, and for generating an output;
• and
• a sixth module for generating real-time outcome-based 3D AR feedback based on the output of the fifth module, and providing said real-time outcome-based 3D AR feedback to the user via the ARUI.