Systems and methods for augmented reality-enhanced field services support

An augmented reality (AR) content generation system, comprising:

• an AR device including a camera, a microphone, a display, and a loudspeaker, the camera being configured to acquire one or more images of a component of a medical imaging or medical therapy device;
• at least one electronic processor programmed to:
• receive, from the microphone, a triggering audio segment;
• generate one or more query data structures from both the one or more images and the triggering audio segment;
• retrieve AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query data structures from a database;
• and
• output the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via the loudspeaker.

An augmented reality (AR) content generation method, comprising:

• acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device;
• receiving, from a microphone of the AR device, a triggering audio segment;
• generating one or more query data structures from both the one or more images and the triggering audio segment by processing the one or more images using a convolutional neural network (CNN) and processing the triggering audio segment using a recurrent neural network (RNN);
• retrieving AR instructional content matching the generated one or more query data structures from a database;
• and
• outputting the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device, the visual instructions including one or more of a relevant section of a service manual related to the component and annotations from the service manual related to the component.