Head-mounted display device and method of operating a display apparatus tracking an object

A non-transitory storage medium containing program instructions that when executed by a processor, cause the processor to perform a method for operating a display apparatus having a display, a camera, and an inertial sensor, the method comprising:

• (a) tracking a pose of an object in a field of view of the camera, the tracking including repeatedly:
• (1) deriving the pose of the object relative to the display apparatus using at least one of:
• image data acquired by the camera and sensor data acquired by the inertial sensor, and (2) displaying, using the display, an image based on the derived pose of the object;
• (b1) deriving first movement of the display apparatus using the sensor data;
• (b2) deriving second movement of the display apparatus or in the field of view using the image data;
• (c) determining whether or not the first movement exceeds a first sensor threshold;
• (d) if the first movement exceeds the first sensor threshold, continuing tracking the pose of the object in step (a);
• (e) if the first movement does not exceed the first sensor threshold:
• (f) determining whether or not the second movement exceeds an image movement threshold;
• (g) if the second movement exceeds the image movement threshold, continuing tracking the pose of the object in step (a);
• and
• (h) if the second movement does not exceed the image movement threshold:
• (1) stopping tracking the pose of the object in step (a);
• and (2) displaying, using the display, the image based on a previously derived pose of the object.

A system comprising:

• a head-mounted display device, including:
• a camera that acquires image data;
• an inertial sensor that acquires sensor data;
• and
• a display;
• and
• a processor or electronic circuit configured to:
• (a) track a pose of an object in a field of view of the camera, the tracking including repeatedly:
• (1) deriving the pose of the object relative to the head-mounted display device using at least one of:
• the image data and the sensor data, and (2) displaying, using the display, an image based on the derived pose of the object;
• (b1) derive first movement of the head-mounted display device using the sensor data;
• (b2) derive second movement of the head-mounted display device or in the field of view using the image data;
• (c) determine whether or not the first movement exceeds a first sensor threshold;
• (d) if the first movement exceeds the first sensor threshold, continue tracking the pose of the object in step (a);
• (e) if the first movement does not exceed the first sensor threshold:
• (f) determine whether or not the second movement exceeds an image movement threshold;
• (g) if the second movement exceeds the image movement threshold, continue tracking the pose of the object in step (a);
• and
• (h) if the second movement does not exceed the image movement threshold:
• (1) stop tracking the pose of the object in step (a);
• and (2) display, using the display, the image based on a previously derived pose of the object.