Method and apparatus for mentoring via an augmented reality assistant

An apparatus for utilizing augmented reality in assisting users comprising:

• an understanding module for generating a scene understanding based on video and audio input of a scene comprising a user performing a task in the scene;
• a correlation module for correlating the scene understanding with a knowledge base to produce a task understanding, comprising one or more goals, of a current activity of the user;
• a reasoning module for reasoning, based on the task understanding and a user's current state, a next step for advancing the user towards completing one of the one or more goals of the task understanding;
• and
• an augmented reality generator for overlaying the scene with an augmented reality view comprising one or more visual and audio representation of the next step of the task to the user,
• wherein the assisting either trains users in completing a task understanding or guides users during operation of the task understanding.

A computer-implemented method for training users or for guiding users during operation comprising:

• accessing a task understanding stored in a knowledge base, comprising one or more goals relating to a current activity of a user;
• deriving a current state of the user based on video and audio input of a scene comprising the user;
• reasoning, based on the task understanding and the user's current state, a next step for advancing the user towards completing one of the one or more goals of the task understanding;
• and
• overlaying a view of the scene for the user with an augmented reality view comprising one or more visual and audio representations of the next step of the task.