System and method for generating asynchronous augmented reality instructions

A method of operating an augmented reality (AR) system, the method comprising:

• capturing first images of a first real-world workspace using a camera of a first head mounted AR device of the AR system being worn by a first user;
• processing the first images using a first processor of the AR system to identify physical objects in the first real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace;
• rendering virtual objects representing the identified physical objects in the 3D space at the respective 3D positions for the identified physical objects;
• displaying the virtual objects on a display of the head mounted AR device;
• recording a first 2D video of the user manipulating a first one of the identified physical objects to perform a first procedural task;
• tagging the first 2D video to the virtual object corresponding to the first one of the physical objects;
• storing the first 2D video in a memory of the AR system;
• capturing second images of a second real-world workspace using a camera of a second head mounted AR device being worn by second user;
• processing the second images using a second processor to identify physical objects in the second real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace;
• and
• rendering a virtual display on the display of the second head mounted AR device for displaying the 2D video.

An augmented reality (AR) system comprising:

• a first head mounted AR device including at least one camera and a display;
• a second head mounted AR device including at least one camera and a display;
• at least one hand-held controller;
• sensors configured to detect positions, movements and orientations of the first head mounted AR device, the second head mounted AR device, and the at least one hand-held controller;
• at least one processing system configured to:
• process first images captured by the camera of the first head mounted AR device to identify physical objects in a first real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace;
• render virtual objects representing the identified physical objects in the 3D space at the respective 3D positions for the identified physical objects on the display of the first head mounted AR device;
• enable manipulation of the virtual objects using the hand-held controllers;
• record the manipulation of the virtual objects as augmented reality content;
• store the augmented reality content in the memory;
• process second images captured by the camera of the second head mounted AR device to identify physical objects in a second real-world workspace and detect 3D positions of the identified physical objects in a 3D space corresponding to the first real-world workspace;
• and
• render the first augmented reality content on the display of the second head mounted augmented reality device to demonstrate performance of the first procedural task.