Methods and devices for extended reality device training data creation

A method for one or more processors to implement in a computing device including the one or more processors and a memory storing original training data for tracking a real object using an extended reality (XR) device, the method comprising:

• receive image frames of the real object acquired by the XR device, the images being acquired when flag data indicated that an XR object displayed on the XR device was consistent in pose with the real object;
• extracting feature data of the real object from the image frames, the feature data including the tracked pose of the real object in the respective image frames;
• and
• generating another training data to replace or update the original training data, the another training data based at least in part on the extracted feature data.

A method for one or more processors to implement in an extended reality (XR) device including a camera, the one or more processors, at least one memory, and a display, the method comprising:

• acquiring, from the camera, a camera data sequence including a first image frame of a real object in a scene;
• tracking a pose of the real object with respect to the camera along the camera data sequence, the pose being derived based at least on the first image frame and original training data generated from at least one of (i) a synthetic image of a 3D model rendered from a predetermined view and (ii) a camera image of a reference real object captured from the view, where the 3D model and the reference real object correspond to the real object;
• displaying an XR object on the display by rendering the XR object based at least on the pose;
• setting flag data, in a memory area of the at least one memory, indicative of whether or not the displayed XR object is consistent in pose with the real object, in response to receipt of an input of a user of the XR device;
• outputting, to a separate computing device having another processor, second image frames in the camera data sequence acquired when the flag data indicates that the displayed XR object is consistent in pose with the real object.

A method for one or more processors to implement in an extended reality (XR) device including a camera, the one or more processors, at least one memory, and a display, the method comprising:

• acquiring, from the camera, a camera data sequence including a first image frame of a real object in a scene;
• tracking a pose of the real object with respect to the camera along the camera data sequence, the pose being derived based at least on the first image frame and original training data generated from at least one of (i) a synthetic image of a 3D model rendered from a predetermined view and (ii) a camera image of a reference real object captured from the view, where the 3D model and the reference real object correspond to the real object;
• displaying an XR object on the display by rendering the XR object based at least on the pose;
• setting flag data, in a memory area of the at least one memory, indicative of whether or not the displayed XR object is consistent in pose with the real object, in response to receipt of an input of a user of the XR device;
• extracting feature data of the real object from second image frames in the camera data sequence acquired when the flag data indicates that the displayed XR object is consistent in pose with the real object, the feature data including the tracked pose of the real object in the respective second image frames;
• and
• generating another training data to replace or update the original training data, the another training data based at least in part on the extracted feature data.