Systems and methods for augmented reality-enhanced field services support

An augmented reality (AR) content generation system, comprising:

• an AR device including a camera, a microphone, a display, and a loudspeaker, the camera being configured to acquire one or more images of a component of a medical imaging or medical therapy device;
• at least one electronic processor programmed to:
• process the acquired one or more images with an image processor to generate one or more processed images;
• receive, from the microphone, a triggering audio segment;
• process the received triggering audio segment with a speech-to-text process or a transcription process to generate a processed triggering audio segment generate one or more query vectors from both the one or more processed images and the processed triggering audio segment;
• retrieve AR instructional content related to the medical imaging or medical therapy device matching the generated one or more query vectors from a database;
• and
• output the AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via the loudspeaker.

An augmented reality (AR) content generation method, comprising:

• acquiring, with a camera of an AR device, one or more images of a component of a medical imaging or medical therapy device;
• receiving, from a microphone of the AR device, a triggering audio segment;
• identifying AR instructional content related to the medical imaging or medical therapy device in a database using both the one or more images of the component of the medical imaging or medical therapy device and the triggering audio segment;
• retrieving the identified AR instructional content from the database;
• and
• outputting the retrieved AR instructional content one or more of (i) displayed superimposed on video displayed by the AR device and/or (ii) displayed on a head mounted display of the AR device and/or (iii) as audio content via a loudspeaker of the AR device, the visual instructions including one or more of a relevant section of a service manual related to the component and annotations from the service manual related to the component.